Summary
User is experiencing issues with incremental sync in Airbyte where updates and inserts from Redshift are not being transferred to Kafka despite the sync being scheduled. Initial sync was successful, but subsequent runs report no records extracted or loaded.
Question
Hi, I have setup Airbyte with Redshift as source and Kafka as destination and have scheduled the sync to run every hour
On the first run, it transferred all the data as expected …Data count was 17000…Post which every hour sync runs …however when the data count at the source changes in case of an insert or update, we are expecting Airbyte should figure the changes and transfer the updates to the destination…In my case the count is increased to 17200…so i am expecting the 200 row of data to be transfer…but it doesnt happens…It stills says
Sync Succeeded 0 Bytes | no records extracted | no records loaded
What should be done to make the sync work properly ?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.