Memory usage issue when ingesting data from MySQL to PostgreSQL

[slack-user-airbyte]

Summary

Data ingestion from MySQL to PostgreSQL is causing memory usage to be full during commit phase

Question

Hii everyone, I am ingesting more than 10GB of data from MySQL to PostgreSQL. It loads all the rows from MySQL, but when it tries to commit, it gets stuck due to full memory usage. Is there any solution for this?

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

_{["mysql", "postgresql", "data-ingestion", "memory-usage", "commit"]}

[slack-user-airbyte]

I was running into a similar issue with Postgres RDS and had to increase the cluster from a t3.xlarge to a t3.2xlarge