Summary
Issues with scaling on Kubernetes using S3 Source and Kafka Destination, slow performance compared to Upsolver configurations
Question
Hey,
We are having issues with scaling on K8s.
We are using S3 Source and Kafka destination.
We configured worker and job resources to 8 CPU’s and 16 GB memory on limit
We have adjusted all of the settings in the Kafka destination connector.
We are getting very slow performance compared to our Upsolver configurations.
Trying to figure out if it’s something with our setup or it’s not intended to high throughput.
Attaching the logs
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.