Sync Stuck in Running / K8s

Connector Questions & Issues
,
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Google GKE
  • Memory / Disk: 4 CPU / 14 GB
  • Deployment: Kubernetes ?
  • Airbyte Version: 0.33.11-alpha
  • Source name/version: Oracle 0.3.14
  • Destination name/version: GCS 0.2.1
  • Step: During Sync
  • Description:

Sometimes, the sync remains stuck in “Running” state.
The job usually takes like 1-2 minutes to execute.

Today it was like that until i hit cancel after approx 10 hours.
After i cancelled the job it worked fine.
Logs enclosed.
logs-32372-1.txt (35.9 KB)

2

Hey @AlexandruManolache1 can you check the memory usage while the sync is running looks like it’s affected because of it. If you can try increasing the memory and sync it again

3

hi @harshith
i now have a job that only syncs around 1000 rows, so very little.
it runs every 3 minutes, and sometimes it stays in Retrying.
The summary sync at the end of the logs specify that the state=failed, but it remains in Retrying
Log Enclosed.

Is there a way i could set a timeout so that the job goes in Failed, so that i can be restarted?
logs-33653-0.txt (18.3 KB)

4

Looks to me like the worker is off and was not running probably. Would suggest increasing the memory and CPU and try again?

5

hi @harshith ,
Thanks for your reply.
The issue with the jobs stuck in Retrying got fixed by restarting the pods.
Now i just have from time to time(every few days) the issue with the job getting stuck in Running.
The only 2 lines i see in logs are:

2022-05-10 03:27:03 INFO WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.33.11-alpha
2022-05-10 03:27:03 INFO TemporalAttemptExecution(get):116 - Executing worker wrapper. Airbyte version: 0.33.11-alpha

and nothing after that.

This could be a memory issue also?

6

Hey also would suggest to upgrade the airbyte version to latest

7

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.