Postgres data extraction issues

Connector Questions & Issues
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: CentOS
  • Memory / Disk: 4GB
    logs-2391.txt (637.6 KB)
    , 200GB
  • Deployment: Kubernetes
  • Airbyte Version: 0.35.62-alpha
  • Source name/version: Postgres 0.4.4, 04.10
  • Destination name/version: Postgres 0.3.17
  • Step: Data extraction from the source
  • Description:
    The goal is to load data from one Postgres DB to another.
    Postgres source connector >= 0.4.5 lead to the issue that no tables fetched from the source. I saw an issue from Feb 15th but no plans from Airbyte team. 🐛 Source Postgres: cannot retrieve all tables accessible to the user · Issue #10649 · airbytehq/airbyte · GitHub
    Source connector 0.4.4 fetches the schema but connection fails with error: Unable to find pod to retrieve exit value. Defaulting to value 143. This is expected if the job was cancelled.
    Logs attached.
2

Are you able to use other source connector to your destination, can I suggest the PokeAPI to validate the pipeline?

3

I have a lot of others connections (Amplitude, Close, Mailchimp, Stripe, etc.) and they work fine with the same target.
I’m already tried to turn off normalization and have the same issues with connector 0.4.4

4

Would be possible to you increase the resources (memory/cpu) available for Airbyte in your k8s cluster?

5

We tried to increase x2 but it didn’t help =(
Interesting fact, that I choose some of tables in 0.4.4 connector and it works fine.

6

Selecting only a subset of table worked?

7

Yes, it’s work.
But previously the same tables failed

1 Like