Source process exited with non-zero exit code 143 for a customized source

Connector Questions & Issues
1
  • OS Version / Instance: Docker
  • Memory / Disk:
    JOB_MAIN_CONTAINER_CPU_LIMIT: “2”
    JOB_MAIN_CONTAINER_CPU_REQUEST: “1”
    JOB_MAIN_CONTAINER_MEMORY_LIMIT: “4Gi”
    JOB_MAIN_CONTAINER_MEMORY_REQUEST: “2Gi”
  • Deployment: Kubernetes
  • Airbyte Version: 0.35.65-alpha
  • Source name/version: Talon One (customized source)
  • Destination name/version: SnowFlake / 0.4.33
  • Description:

Hey all, I created a customized source for our business development. The local testing from Airbyte read command worked fine without any issues. However, after setting up a connector between the customized source and the built-in SnowFlake destination, the initial sync can populate an error as Source process exited with non-zero exit code 143, if the initial sync needs to convey a considerably large amount of data (584.59 MB) to the destination. When I went over the log, it was unclear what caused this. I attached the log file below. Thanks all!
logs-40385.txt (260.7 KB)

2

Hey @Da_Wang, thanks for your post and welcome to the community!

Firstly, would you be able to increase the memory of the JOB_MAIN_CONTAINER_MEMORY_REQUEST to something like 4gb or 8gb?

And then post the sync logs of that sync.

3

Hi there, I also have this issue, some kind of memory leak is going on.

My connector succeeds, and then the worker does not clear memory so subsequent runs will fail.

4

Hi @Kevin_Han, we’ve been getting reports from other users about similar behavior. The team is investigating it and we’re looking into it. Hope you can bear with us.