MSSQL -> BigQuery behaving strangely

  • Is this your first time deploying Airbyte?: No, been using Airbyte for more than a year.
  • OS Version / Instance: [debian-10-buster-v20220406]
  • Memory / Disk: GCP e2-highmem-4, with 100GB boot disk
  • Deployment: Docker
  • Airbyte Version: 0.38.1-alpha
  • Source name/version: airbyte/source-mssql 0.3.21
  • Destination name/version: airbyte/destination-bigquery 1.1.4
  • Step: Sync

Syncs are failing every now and then for some reason.

What really causes problems here is that Airbyte overwrites the destination table with 0 rows. It would be better if it didn’t touch the destination table. Best would of course be if it worked :slight_smile:

Attaching logs for one failed sync below:
logs-166.txt (521.7 KB)

It’s the same issue as reported here : Destination BigQuery : 400 Bad Request · Issue #12813 · airbytehq/airbyte · GitHub

1 Like

Thank you @anatole ! Have you been able to find a working BigQuery destination connector version?

Hi @maxkrog ,
Unfortunately not.
It’s possible that the issue is on google side though.

@maxkrog are you using standard sync or gcs staging?

I was using standard sync. Testing with GCS staging now. @marcosmarxm

Max please update if issues continues after changing to GCS staging

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.