Sync Issue with Data Transfer from AWS Redshift to Kafka Topics


Issue with sync job not transferring expected data from AWS Redshift to Kafka Topics


We’ve implemented Airbyte on our Kubernetes cluster to facilitate data transfer from AWS Redshift to Kafka Topics. While we’ve achieved successful data transmission from the source to the destination, we’ve encountered an issue where sync doesn’t work as expected.
We are expecting that when a sync job runs if there is any difference in the data in source and destination it should transfer the data…However it says 0 records where source contains lets say 1,00,000 data and destination has 9,95,000
Any assistance or insights on resolving this matter would be greatly appreciated.

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["sync-issue", "data-transfer", "aws-redshift", "kafka-topics"]