High Disk IOPS on Airbyte Docker Compose Setup on EC2

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

Investigating high disk IOPS saturation on Airbyte running in Docker Compose on EC2 instance.


Question

We’re running airbyte using docker compose (still) on EC2 machine. Our box is locking up due to saturating disk IOPS 3k+. We’re using mostly default configuration, anyone know why this might be happening? Or even better, how to resolve or troubleshoot why?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["high-disk-iops", "docker-compose", "ec2", "troubleshoot"]
2

would be nice if abctl was viable option, but so far we’ve had 0 luck getting it working locally. :cry: Docs are missing too, so hard to configure it to use RDS on a EC2 box.

Docker compose is very straight forward and easy to configure, shame its failing so badly at the moment. we’re restarting our box almost daily now due to disk IOPS saturation, and can’t find the cause.

3

The official solution is migrate to abctl :laughing::octavia-thanks: