Hi! Having trouble with a Snowflake <> Snowflake sync on Airbyte Cloude during a large data pull from the source. Connection is closed unexpectedly at the same point on every retry, at roughly 26GB synced.
- Is this your first time deploying Airbyte?: Using Airbyte Cloud
- OS Version / Instance: Airbyte Cloud
- Memory / Disk: Airbyte Cloud
- Deployment: Airbyte Cloud
- Airbyte Version: 0.36.0-alpha-cloud
- Source name/version: Snowflake (0.1.10)
- Destination name/version: Snowflake (0.4.24)
- Step: Initial Sync
- Description: Sync fails after 26GB pulled from destination (3x failure at the same point). Source & destination are using a Large compute instance in Snowflake. Sync works when tried with less data (<1GB, same source, destination, and schema).
My logs: logs-133156.txt (7.0 MB)
Hi @SpicyBajo,
Could you please reach our technical support team if you encounter issues on Airbyte Cloud? They are reachable on the online chat that you’ll find in the web console of your workspace.
This could be a resource management problem. It could also be useful if you could track what’s happenning on Snowflake side during the sync too.
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.