Snowflake destination connection dropping during large data sync

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Issue with Snowflake connector dropping connection during large data sync from MySQL to Snowflake. Occurs only on longest running sync, smaller syncs work fine. Seeking suggestions to prevent connection drops.


Question

Snowflake destination problem. I am running into an issue when syncing large amounts of data from mysql to snowflake. We have lots of connections that are working successfully. However, this one seems to be failing pretty consistently.
Airbyte version 0.50.56
Snowflake connector 3.2.3

This is in our production environment so havent gotten to upgrade versions yet but wanted to reach out here to see if anyone had run into anything similar. Only seems to happen on this longest running sync. Smaller syncs from the same source work fine. Anyone have any suggestions on what could be causing the connection to drop? Or ways to keep this from happening?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["snowflake-connector", "mysql", "large-data-sync", "connection-drop", "production-environment"]
2

stacktrace error excerpts

...
2024-06-27 20:48:04 destination >         Caused by: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
...
2024-06-27 20:48:04 destination >         Caused by: java.io.EOFException: SSL peer shut down incorrectly
2024-06-27 20:48:04 destination >                 at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489) ~[?:?]
2024-06-27 20:48:04 destination >                 at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:478) ~[?:?]
2024-06-27 20:48:04 destination >                 at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160) ~[?:?]
2024-06-27 20:48:04 destination >                 at sun.security.ssl.SSLTransport.decode(SSLTransport.java:111) ~[?:?]
2024-06-27 20:48:04 destination >                 at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1510) ~[?:?]
2024-06-27 20:48:04 destination >                 at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1425) ~[?:?]
2024-06-27 20:48:04 destination >                 at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) ~[?:?]
2024-06-27 20:48:04 destination >                 at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426) ~[?:?]```