Summary
The user is facing an issue with the MySQL CDC connector while trying to sync data from a MySQL table to S3 in a CDC (incremental refresh) manner. They are encountering a ‘java.io.IOException: Socket is closed’ error. They also have queries regarding skipping the initial snapshot of the table and only picking CDC logs, as well as insights on the error causing data to be picked from the start after 10 hours of execution.
Question
Hi All,
We were exploring the mysql CDC connector to sync data from a mysql table to s3 in cdc (incremental refresh manner).
As per the documentation, we checked for the first time it will perform the full dump (by executing select statements)
But while fetching we were facing the issue
java.net.SocketException
java.net.SocketException: Connection reset by peer```
details stacktrace is mentioned in reply on the same thread.
I have two queries :
1. Is there any way we can only pick cdc logs [like manually setting the offset in airbyte connection state?] and skip reading/making initial snapshot of the table (as some tables are really huge and reading data take more hours and we have other process to have one time dump of the tables, so don't want to do rework).?
Found a thread on the same <https://discuss.airbyte.io/t/syncing-huge-tables-10b-rows-from-mysql/3783>
<@U02T7NVJ6A3> Any idea plans to have this feature in the dev pipeline?
1. Any insights on the above error [we got the error after 10 hrs of execution on retry it again starts picking data from the start as per the logs]?
Any help would be much appreciated..
Thanks
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1707302760814389) if you want to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
["mysql-cdc-connector", "incremental-sync", "s3", "java-io-exception", "cdc-logs", "initial-snapshot", "error"]
</sub>