Mongodb connector failure

logs-491.txt (243.6 KB)

  • Is this your first time deploying Airbyte?: No
  • Instance: GCP/Debian GNU/Linux 10 (buster)/e2-standard-2
  • Memory/Disk: 8GB/200GB
  • Deployment: Docker
  • Airbyte Version: 0.35.65-alpha
  • Source name/version: MongoDB 0.1.13
  • Destination name/version: BigQuery 1.0.2
  • Step: Sync
  • Description: The sync hangs after reaching 1GB of data and then fails after 14 hours. No data is transferred.

Hi @matfiorlh,
The following error usually characterizes a lack of memory resources:
DEADLINE_EXCEEDED: deadline exceeded after 69.999395343
I would suggest you upsize your GCP instance to a least 16GB memory.

1 Like

Thanks @alafanechere, I’ll give it a try. Is there an easy way to size the VM?

We have some guidance here: Scaling Airbyte | Airbyte Documentation
It really depends of your sources data shape and of the number of connections you run on Airbyte.