- Is this your first time deploying Airbyte?: No
- OS Version / Instance: Ubuntu
- Memory / Disk: 32Gb / 30 Gb
- Deployment: Docker
- Airbyte Version: 0.39.41-alpha
- Source name/version: MySQL (0.6.1)
- Destination name/version: BigQuery (1.1.14)
- Step: The issue is happening during sync.
- Description: After 3 syncs the job fails and no error shows up in the logs.
The last item on the log is
2022-08-08 00:36:32 e[32mINFOe[m i.a.w.t.TemporalUtils(withBackgroundHeartbeat):291 - Stopping temporal heartbeating...
Not sure why and there’s enough disk space and no error on GCP or limit issues. After analysing the previous successful logs I realised it fails when starting up the normalisation process.
Can you share the complete sync log?
Are you using a full refresh or incremental sync?
After the 3rd sync always the sync fails?
How many tables are you trying to sync to BigQuery and how large is the sync (in Gb)?
Does the server have only 30Gb of disk? (can you check with docker system df)
Its an incremental sync | deduped and history
There are 82 tables i’m trying to sync to bigQuery
Total size in sync is 580Mb
It fails sometimes after 3 or 4 syncs
Can you share the complete log, is it not possible to help you without it.