Low memory footprint - how to set job main container memory parameters?

Hi @kyle-mackenzie-indee,
These values should have the same structure as those usually defined in docker-compose files for memory request.
You can set JOB_MAIN_CONTAINER_MEMORY_REQUEST=28g .
I’m not sure that this will improve the throughput of you postgres sync though. I would suggest you upgrade your source postgres connector to the latest version. We recently improved this connector to have a dynamic fetch size that was previously hardcoded to 1000 records.
You’ll find an interesting related discussion here about source database performance. The discussion is around MySQL but the core logic is exactly the same for Postgres.

1 Like