Is this your first time deploying Airbyte : No OS Version / Instance: EC2 t2.large Deployment : EC2 Docker Airbyte Version : 0.35.4-alpha Source name/version : Postgres 0.4.14 Destination name/version : BigQuery 1.0.1 Description : BigQuery destination fails to close and causes sync failures
Hi,
Since this morning my Postgres to BQ connection is failing because of the below error. Strangely the other connections to BQ are not failing even though the error seems to be BQ related.
It is an Incremental Dedup connection with the standard Loading Method.
Note that I have the same error after upgrading the destination connector to BigQuery 1.1.4
It’s happening approx 75% of the time. And it corrupts the destination data because the cursor gets updated but not the data in the destination so the data since the first occurence of this bug is lost.
Is there any way to manually override the value of the cursor ?
Thanks
Hey ! Since this night, I encounter an issue almost similar (throwing a JSON schema validation failed error) for my postgreSQL <-> BigQuery connection.
I tried to re-run many times but all my jobs failed since then. It’s odd because I did not make any update on my stack since at least a week so I don’t see what could be the cause.
Not sure if it’s the same error or not but as I first encountered this issue last night and @anatole posted it a few hours ago (+ both logs mention JSON validation error) , I felt it might exits a link between our issues so I thought my logfile could add some input to find a solution.
I have/had the same issue. Airbyte is behaving very strangely and the behaviour of Full Refresh | Overwrite syncs are not consistent, even when source data is identical.