Summary
When facing issues with record mismatches in Airbyte sync due to infrastructure, how to backfill and sync specific records without running a full refresh. Avoiding manual scripting.
Question
I have a question regarding the sync of Airbyte. We have implemented a quality check that counts the number of records in the source and destination and compare them. The numbers matched most of the time but rarely it mismatched. We traced this problem and might be an issue with out infrastructure hosting Airbyte not with Airbyte itself.
The problem here is that some records wasnt synced successfully. How to backfill and sync these specific records again with running a full refresh which is quite expensive. Is it possible to just sync a couple of records or a specific day perhaps?
We dont want to have to go through writing a manual script that will handle these records on its own which will be like what Airbyte does. Any ideas here.
Thanks in advance
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.