MSSQL -> BigQuery behaving strangely

Troubleshooting
,
#1
  • Is this your first time deploying Airbyte?: No, been using Airbyte for more than a year.
  • OS Version / Instance: [debian-10-buster-v20220406]
  • Memory / Disk: GCP e2-highmem-4, with 100GB boot disk
  • Deployment: Docker
  • Airbyte Version: 0.38.1-alpha
  • Source name/version: airbyte/source-mssql 0.3.21
  • Destination name/version: airbyte/destination-bigquery 1.1.4
  • Step: Sync

image
image991×817 55.5 KB

image
image991×817 93.1 KB

Syncs are failing every now and then for some reason.

image
image1020×817 87.5 KB

What really causes problems here is that Airbyte overwrites the destination table with 0 rows. It would be better if it didn’t touch the destination table. Best would of course be if it worked :slight_smile:

Attaching logs for one failed sync below:
logs-166.txt (521.7 KB)

#2

It’s the same issue as reported here : Destination BigQuery : 400 Bad Request · Issue #12813 · airbytehq/airbyte · GitHub

1 Like
#3

Thank you @anatole ! Have you been able to find a working BigQuery destination connector version?

#4

Hi @maxkrog ,
Unfortunately not.
It’s possible that the issue is on google side though.

#5

@maxkrog are you using standard sync or gcs staging?

#6

I was using standard sync. Testing with GCS staging now. @marcosmarxm

#7

Max please update if issues continues after changing to GCS staging