Summary
When starting replication in Airbyte from Postgres source to Postgres destination, memory limit set on Docker containers works well initially. However, dynamically started Postgres source and destination containers consume all RAM, causing the instance to freeze. User wants to limit each Postgres container to 4GB and also set the same limit for the normalization container that starts dynamically at the end.
Question
Hello All, I am using airbyte for postgres src to postgres dest
I am on 16gb ram server
when i start the replication, i set mem_limit on all docker container and it is working well
but once sync starts, postgres src and dest containers start , since they start dynamically i am not able to set memory limit on them
due to which they end up eating all the ram and instance gets freezed
I am ok if sync happens slowly but i want postgres src and dest container to use not more than 4gb each
and same for normalisation containerwhich starts dynamically at end
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.