Summary
Experiencing MongoDB to BigQuery data transfer timeout errors after switching from Airbyte Cloud to self-hosted Airbyte on AWS. Transfer fails consistently with ~1 million documents from MongoDB. Stuck at 1.2 million documents with a different collection of ~1.3 million documents.
Question
Hi team,
I’m experiencing issues with my MongoDB to BigQuery data transfers. Previously, I was using Airbyte Cloud for this transfer, and it worked seamlessly. However, after hosting the Airbyte open-source version on AWS, I’m facing repeated MongoDB timeout errors.
Here are the details:
• Source: MongoDB, with a stream of ~1 million documents.
• Error: Transfer fails each time with a timeout error.
• Other attempts: I tried with a different collection of ~1.3 million documents. In this case, it gets stuck consistently at 1.2 million documents before timing out.
I’ve attached a screenshot for reference. Could you help me resolve this?
Thanks!
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.