Sync Stuck in Running / K8s

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Google GKE
  • Memory / Disk: 4 CPU / 14 GB
  • Deployment: Kubernetes ?
  • Airbyte Version: 0.33.11-alpha
  • Source name/version: Oracle 0.3.14
  • Destination name/version: GCS 0.2.1
  • Step: During Sync
  • Description:

Sometimes, the sync remains stuck in “Running” state.
The job usually takes like 1-2 minutes to execute.

Today it was like that until i hit cancel after approx 10 hours.
After i cancelled the job it worked fine.
Logs enclosed.
logs-32372-1.txt (35.9 KB)

Hey @AlexandruManolache1 can you check the memory usage while the sync is running looks like it’s affected because of it. If you can try increasing the memory and sync it again

hi @harshith
i now have a job that only syncs around 1000 rows, so very little.
it runs every 3 minutes, and sometimes it stays in Retrying.
The summary sync at the end of the logs specify that the state=failed, but it remains in Retrying
Log Enclosed.

Is there a way i could set a timeout so that the job goes in Failed, so that i can be restarted?
logs-33653-0.txt (18.3 KB)

Looks to me like the worker is off and was not running probably. Would suggest increasing the memory and CPU and try again?

hi @harshith ,
Thanks for your reply.
The issue with the jobs stuck in Retrying got fixed by restarting the pods.
Now i just have from time to time(every few days) the issue with the job getting stuck in Running.
The only 2 lines i see in logs are:

2022-05-10 03:27:03 INFO WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.33.11-alpha
2022-05-10 03:27:03 INFO TemporalAttemptExecution(get):116 - Executing worker wrapper. Airbyte version: 0.33.11-alpha

and nothing after that.

This could be a memory issue also?

Hey also would suggest to upgrade the airbyte version to latest