Snowflake Sync Source Failure after 26GB, Airbyte Cloud

Troubleshooting
#1

Hi! Having trouble with a Snowflake <> Snowflake sync on Airbyte Cloude during a large data pull from the source. Connection is closed unexpectedly at the same point on every retry, at roughly 26GB synced.

  • Is this your first time deploying Airbyte?: Using Airbyte Cloud
  • OS Version / Instance: Airbyte Cloud
  • Memory / Disk: Airbyte Cloud
  • Deployment: Airbyte Cloud
  • Airbyte Version: 0.36.0-alpha-cloud
  • Source name/version: Snowflake (0.1.10)
  • Destination name/version: Snowflake (0.4.24)
  • Step: Initial Sync
  • Description: Sync fails after 26GB pulled from destination (3x failure at the same point). Source & destination are using a Large compute instance in Snowflake. Sync works when tried with less data (<1GB, same source, destination, and schema).

My logs: logs-133156.txt (7.0 MB)

#2

Hi @SpicyBajo,
Could you please reach our technical support team if you encounter issues on Airbyte Cloud? They are reachable on the online chat that you’ll find in the web console of your workspace.
This could be a resource management problem. It could also be useful if you could track what’s happenning on Snowflake side during the sync too.