Error during GitLab to Snowflake sync

Summary

User reports frequent retries and failures while syncing data from GitLab to Snowflake, with specific error logs indicating job failures after multiple retries and missing status in the orchestrator pod.


Question

Hi Team,
I am syncing gitlab repo to snowflake. I often get below error which causes lot of retries . Any pointer would be appreciated .

Running on : EKS
version : 0.50.34


Pool queue size: 0, Active threads: 0

2023-12-18 02:11:45 INFO i.a.w.p.AsyncOrchestratorPodProcess(computeExitValue):190 - The current non terminal state is RUNNING

2023-12-18 02:11:51 INFO i.a.w.t.s.a.AppendToAttemptLogActivityImpl(log):56 - Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=10, successivePartialFailureLimit=1000, totalPartialFailureLimit=10, successiveCompleteFailures=5, totalCompleteFailures=5, successivePartialFailures=0, totalPartialFailures=0)

Backoff before next attempt: 13 minutes 30 seconds

2023-12-18 02:11:51 ERROR i.a.w.t.s.a.AppendToAttemptLogActivityImpl(log):54 - Failing job: 482491, reason: Job failed after too many retries for connection 9c76285b-f1e6-4811-bddb-1c61ecd2e581

2023-12-18 02:11:50 WARN i.a.w.p.AsyncOrchestratorPodProcess(computeExitValue):198 - State Store missing status, however orchestrator pod orchestrator-repl-job-482491-attempt-4 in terminal. Assume failure.```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1702870697972809) if you want
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
['gitlab', 'snowflake', 'sync-error', 'retry', 'eks', 'job-failure']
</sub>