Data Loss Issue with Full Refresh - Overwrite Sync Mode


When syncing data from Redshift to MySQL using Full Refresh - Overwrite mode, the destination table was erased during a sync failure due to connection failure. Questioning if the data loss issue is related to the MySQL connector behavior or if there are other overlooked issues.


Hi all, recently I encountered an issue that the destination table has been erased while the sync failed due to connection failure (authentication failed). I am syncing data from Redshift to MySQL using Full Refresh - Overwrite mode. I expect the data in the destination table will remain the same as it was in the last sync if there is any sync failure. During my research, I have found something mentioned in the Airbyte docs that some connector with Full-Refresh - Overwrite mode may indeed need to erase the existing data at the start of each full-refresh sync ( but it doesn’t explicitly mention if MySQL connector falls in this case. I wanna know if the data lost issue is related to the connector OR I may have overlooked others issues. Thanks and much appreciated if anyone can give my some directions.

Airbyte: 0.50.19
MySQL: v0.2.0
Redshift: v0.4.0

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["data-loss", "full-refresh-overwrite", "redshift", "mysql-connector", "sync-failure"]