Postgres standard replication to Bigquery hangs without error

Connector Questions & Issues
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Almalinux
  • Memory / Disk: you can use something like 32Gb / 1 Tb
  • Deployment: Docker
  • Airbyte Version: 0.39.23
  • Source name/version: Postgres 0.4.26
  • Destination name/version: Bigquery 1.1.9
  • Step: Sync
  • Description:
    Using a Postgres standard replication to Bigquery hangs for hours without sending an error to the log. The last log message reads:
    INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):107 - Preparing tmp tables in destination completed.
2

Hi @oldfaust,
Could you please share your full sync logs please? It will help us debugging this problem and try to understand if the problem is on the source or destination side.
Could you also please share the available memory on your Airbyte memory while running the sync?
Finally, please double check the users you use to connect to Postgres / Bigquery has sufficient permissions to read (for source) and write (for destination).

3

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.