Scaling airbyte-worker pods for resource optimization


Optimizing resource consumption by scaling airbyte-worker pods and spreading out loads


Hello guys, I have a question that maybe some of you have already had

What is the best way to scale airbyte-worker pods? I only have 2 of these pods running but they are consuming a lot of resources, like 4 vCPUs and 4GB RAM each. Would be nice if the worker scale in a way that each pod doesn’t consume so many resources alone

Is there a way to spread out theses loads across multiples pods?

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["scaling", "airbyte-worker", "resource-optimization", "pods", "resource-consumption"]