Bigquery details: 400 Bad Request

Connector Questions & Issues
1
  • Is this your first time deploying Airbyte?: No
  • OS Version: debian-10
  • Memory / Disk: 16GB
  • Deployment: Docker
  • Airbyte Version: 0.36.1-alpha
  • Source name/version: Hubspot 0.1.51
  • Destination name/version: BigQuery 1.1.4
  • Step: Loading Method GCS staging doesn’t run
  • Description:
    Insert into BigQuery fails. I noticed that if I reduce the number of tables that are fetched it works just fine so I suspect that it has something to do with the way records are staged. Thus, I tried changing the insert method but that gave me another error.
    Can you please advice?
    logs-914 (1).txt (322.8 KB)
2

Hi @i-zaykov ,
It seems that we have the same issue : Postgres to BigQuery Sync fails - #4 by anatole
I’ll let you know if I manage to fix it. Please do the same

3

Thanks will do @anatole

4

ivan the log error are you using staging or standard insert?
A few users reported the same issue with BigQuery

2022-05-12 11:25:06 e[43mdestinatione[0m > ... 9 more2022-05-12 11:25:06 e[43mdestinatione[0m > "errors": [2022-05-12 11:25:06 e[43mdestinatione[0m > {2022-05-12 11:25:06 e[43mdestinatione[0m > "message": "Invalid credential",2022-05-12 11:25:06 e[43mdestinatione[0m > "domain": "global",2022-05-12 11:25:06 e[43mdestinatione[0m > "reason": "invalid"2022-05-12 11:25:06 e[43mdestinatione[0m > }2022-05-12 11:25:06 e[43mdestinatione[0m >],2022-05-12 11:25:06 e[43mdestinatione[0m > "status": "INVALID_ARGUMENT"2022-05-12 11:25:06 e[43mdestinatione[0m > }2022-05-12 11:25:06 e[43mdestinatione[0m > }

The best way to handle this so far is to use GCS staging for long syncs

5

thanks @marcosmarxm but I’m also having problems with GCS sync
BigQuery GCS staging failed - Troubleshooting - Airbyte

6

@ivan are you still having problem with this? Looks people return to normality: https://github.com/airbytehq/airbyte/issues/12813

7

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.