Issue with Binlog-based sync getting stuck at schema history building

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

The user is facing an issue with the binlog-based sync in Airbyte Cloud, where the sync gets stuck at the schema history building step and eventually times out. The user has tried increasing the wait time but the issue persists.


Question

Hi All,

I am using Airbyte Cloud to move data from MySQL 5.7 to MySQL 8. I am new to airbyte and just testing the CDC using binlog capability. While I am able to setup the connection and the sync starts, it gets stuck at one step INFO main i.a.c.i.d.i.RecordWaitTimeUtil(getFirstRecordWaitTime):68 First record waiting time: 1200 seconds
I increased the wait time from 300 to 1200 and it is still the same. After a while the sync is failing with the error Java.lang.RuntimeException: java.lang.RuntimeException: Building schema history has timed out. Please consider increasing the debezium wait time in advanced options.
I would appreciate any help on this.

Note - I was able to make the cursor-based sync work. Binlog-based sync is where it is failing


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["airbyte-cloud", "binlog-based-sync", "CDC", "MySQL-5.7", "MySQL-8", "schema-history", "debezium"]
2

The issue was that the database had many tables, and Airbyte was trying to read everything during the sync even though the sync was set up only for one table. The issue was resolved after we restricted access to only the required tables for the Airbyte user. Any idea why it was trying to read the whole database?