MySQL CDC Connector Failing After Source Database Downtime

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

The MySQL CDC connector is failing after the source database downtime due to a missing binary log index. The user needs help in resolving this without having to reset and reload the full data.


Question

Hi Team,
we have scheduled a mysql cdc connector and it is running fine.
But in between my source database was down for sometime. After this the job is failing with exception

2024-02-22 06:16:25 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource.wrap(MySqlStreamingChangeEventSource.java:1254) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-02-22 06:16:25 source >         at io.debezium.connector.mysql.MySqlStreamingChangeEventSource$ReaderThreadLifecycleListener.onCommunicationFailure(MySqlStreamingChangeEventSource.java:1299) ~[debezium-connector-mysql-2.4.0.Final.jar:2.4.0.Final]
2024-02-22 06:16:25 source >         at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1079) ~[mysql-binlog-connector-java-0.28.1.jar:0.28.1]
2024-02-22 06:16:25 source >         at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:631) ~[mysql-binlog-connector-java-0.28.1.jar:0.28.1]
2024-02-22 06:16:25 source >         at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:932) ~[mysql-binlog-connector-java-0.28.1.jar:0.28.1]
2024-02-22 06:16:25 source >         at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
2024-02-22 06:16:25 source > Caused by: com.github.shyiko.mysql.binlog.network.ServerException: Could not find first log file name in binary log index file
2024-02-22 06:16:25 source >         at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:1043) ~[mysql-binlog-connector-java-0.28.1.jar:0.28.1]
2024-02-22 06:16:25 source >         ... 3 more```
As per the stacktrace , it is pointing to a missing binary log index
Need help on resolving the same (do not want to rest and reload the full data again).

Any help would be much appreciated..

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1708583044586019) if you want to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["mysql-cdc-connector", "source-database", "binary-log-index", "resolve-issue"]
</sub>
2

What is your binlog retention set at?

3

<@U021P0L72CV>
It is set to binlog_expire_logs_seconds
2592000

4

well thats about all i could check :slightly_smiling_face: If the db was down and the instance type changed or was recreated (not sure where this is hosted) you’ll lose the binlog and have to restart

5

this has happened to me a few times