Summary
The user is experiencing high CPU utilization on an EC2 instance running Airbyte OSS with Jira and LinkedIn connectors. They are looking for options to reduce CPU usage, limit memory and CPU at the worker level, and recommendations for CPU and memory configurations for different containers.
Question
Hello there,
I deployed Airbyte OSS on EC2(m6i.xlarge) and created two connectors(Jira, LinkedIn).
I see CPU utilizations is over 99% when both connectors are syncing and over 50% when only one is syncing.
• Apart from using a larger compute, are there any other options?
• How to limit memory and CPU at worker level?
• What are the recommended CPU and memory configs for the different containers?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.