Hey!
I think you first need to identify the bottleneck.
I gave some insights on a pretty similar question here. The fetchsize
could be the bottleneck on the source size.
I could also suggest you try the E2E testing source and destination connector to check the best throughput you could achieve on the destination and on the source side.
Any tips on how to tune airbyte especially on k8s?
If you realize your bottleneck is resource-related you can change the following environment variables to give more resources to pod running sync:
JOB_MAIN_CONTAINER_CPU_REQUEST=JOB_MAIN_CONTAINER_CPU_LIMIT=JOB_MAIN_CONTAINER_MEMORY_REQUEST=JOB_MAIN_CONTAINER_MEMORY_LIMIT=