Sync Stuck in Running / K8s

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Google GKE
  • Memory / Disk: 4 CPU / 14 GB
  • Deployment: Kubernetes ?
  • Airbyte Version: 0.33.11-alpha
  • Source name/version: Oracle 0.3.14
  • Destination name/version: GCS 0.2.1
  • Step: During Sync
  • Description:

Sometimes, the sync remains stuck in “Running” state.
The job usually takes like 1-2 minutes to execute.

Today it was like that until i hit cancel after approx 10 hours.
After i cancelled the job it worked fine.
Logs enclosed.
logs-32372-1.txt (35.9 KB)

Hey @AlexandruManolache1 can you check the memory usage while the sync is running looks like it’s affected because of it. If you can try increasing the memory and sync it again

hi @harshith
i now have a job that only syncs around 1000 rows, so very little.
it runs every 3 minutes, and sometimes it stays in Retrying.
The summary sync at the end of the logs specify that the state=failed, but it remains in Retrying
Log Enclosed.

Is there a way i could set a timeout so that the job goes in Failed, so that i can be restarted?
logs-33653-0.txt (18.3 KB)

Looks to me like the worker is off and was not running probably. Would suggest increasing the memory and CPU and try again?

hi @harshith ,
Thanks for your reply.
The issue with the jobs stuck in Retrying got fixed by restarting the pods.
Now i just have from time to time(every few days) the issue with the job getting stuck in Running.
The only 2 lines i see in logs are:

2022-05-10 03:27:03 INFO WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.33.11-alpha
2022-05-10 03:27:03 INFO TemporalAttemptExecution(get):116 - Executing worker wrapper. Airbyte version: 0.33.11-alpha

and nothing after that.

This could be a memory issue also?

Hey also would suggest to upgrade the airbyte version to latest

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.