Bigquery details: 400 Bad Request

  • Is this your first time deploying Airbyte?: No
  • OS Version: debian-10
  • Memory / Disk: 16GB
  • Deployment: Docker
  • Airbyte Version: 0.36.1-alpha
  • Source name/version: Hubspot 0.1.51
  • Destination name/version: BigQuery 1.1.4
  • Step: Loading Method GCS staging doesn’t run
  • Description:
    Insert into BigQuery fails. I noticed that if I reduce the number of tables that are fetched it works just fine so I suspect that it has something to do with the way records are staged. Thus, I tried changing the insert method but that gave me another error.
    Can you please advice?
    logs-914 (1).txt (322.8 KB)

Hi @i-zaykov ,
It seems that we have the same issue : Postgres to BigQuery Sync fails - #4 by anatole
I’ll let you know if I manage to fix it. Please do the same

Thanks will do @anatole

ivan the log error are you using staging or standard insert?
A few users reported the same issue with BigQuery

2022-05-12 11:25:06 e[43mdestinatione[0m > ... 9 more2022-05-12 11:25:06 e[43mdestinatione[0m > "errors": [2022-05-12 11:25:06 e[43mdestinatione[0m > {2022-05-12 11:25:06 e[43mdestinatione[0m > "message": "Invalid credential",2022-05-12 11:25:06 e[43mdestinatione[0m > "domain": "global",2022-05-12 11:25:06 e[43mdestinatione[0m > "reason": "invalid"2022-05-12 11:25:06 e[43mdestinatione[0m > }2022-05-12 11:25:06 e[43mdestinatione[0m >],2022-05-12 11:25:06 e[43mdestinatione[0m > "status": "INVALID_ARGUMENT"2022-05-12 11:25:06 e[43mdestinatione[0m > }2022-05-12 11:25:06 e[43mdestinatione[0m > }

The best way to handle this so far is to use GCS staging for long syncs

thanks @marcosmarxm but I’m also having problems with GCS sync
BigQuery GCS staging failed - Troubleshooting - Airbyte

@ivan are you still having problem with this? Looks people return to normality: https://github.com/airbytehq/airbyte/issues/12813