Hi all, our company is currently trying to set up its data warehouse using Airbyte, we are planning on deploying Airbyte on a t3. medium EC2, we have to configure almost 150 MYSQL server sources that need to fetch data using CDC. How much instance load and memory usage is expected if we are planning to set up these many sources? It would be really helpful if you guys can pitch in on the load metrics expectation for running Airbyte on a self-hosted EC2 server
2 Likes
@varun-s-collpoll can you explain your query in a little bit more detail.
Do you mean connecting airbyte to stream data from 150 different MYSql schemas?, in that case how much computation and size of server is required to run the above pipeline?
According to the Airbyte docs, it is recommended to run on a x.large server on AWS, not sure how many connectors and sources are they recommending here!