- Is this your first time deploying Airbyte?: No
- OS Version / Instance: AWS EKS 1.20
- Deployment: kubernates using helm chart
- Airbyte Version: 0.39.17
- Source name/version: MongoDb / 0.1.14
- Destination name/version: S3 / 0.3.6
- Step: The issue is happening during sync
- Description: The schedule that had been going well until now stopped suddenly. To be exact, it’s not stopped, but it doesn’t proceed any more while waiting for the pod. Airbyte and other connectors worked well without any problems and there were no other problems.
look at the sync history, it does not remain failed, but the sync is still working. I looked at the log, and the last thing I did was wait for the pod to work. What problems can cause this and how can it be prevented?
2022-06-28 15:58:36 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Cloud storage job log path: /workspace/192/0/logs.log
2022-06-28 15:58:36 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.39.17-alpha
2022-06-28 15:58:36 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = destination-s3-check-192-0-qcbyl for airbyte/destination-s3:0.3.6
2022-06-28 15:58:36 INFO i.a.w.p.KubeProcessFactory(create):103 - destination-s3-check-192-0-qcbyl stdoutLocalPort = 9026
2022-06-28 15:58:36 INFO i.a.w.p.KubeProcessFactory(create):106 - destination-s3-check-192-0-qcbyl stderrLocalPort = 9027
2022-06-28 15:58:36 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):602 - Creating stderr socket server...
2022-06-28 15:58:36 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):584 - Creating stdout socket server...
2022-06-28 15:58:36 INFO i.a.w.p.KubePodProcess(<init>):515 - Creating pod destination-s3-check-192-0-qcbyl...
2022-06-28 15:58:36 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):305 - Waiting for init container to be ready before copying files...
2022-06-28 15:58:36 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):309 - Init container present..
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):312 - Init container ready..
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(<init>):546 - Copying files...
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: source_config.json
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/e9f7df37-cb38-45cf-80d6-ab5c89a33dde/source_config.json airbyte/destination-s3-check-192-0-qcbyl:/config/source_config.json -c init
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: FINISHED_UPLOADING
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/5dfc545b-61b3-40d5-a669-5f5d61c621be/FINISHED_UPLOADING airbyte/destination-s3-check-192-0-qcbyl:/config/FINISHED_UPLOADING -c init
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process
2022-06-28 15:58:42 INFO i.a.w.p.KubePodProcess(<init>):549 - Waiting until pod is ready...
Hey @zigbang-yeezy, thanks for your post and welcome to the community. The error message from kubernetes is a bit vague so it’s hard to pinpoint exactly what the issue might be. Were you using following this guide: https://docs.airbyte.com/deploying-airbyte/on-kubernetes/?
Can you post the full server logs and have you tried upgrading to the latest airbyte version?
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.