Summary
The user is looking for a way to handle deletes in incremental sync with Airbyte when soft delete flag or CDC is not an option. They also want to set up a monthly full refresh for the tables.
Question
howdy Airbytians - I have an interesting question for the community today. I have two tables in a single sync that I have set up to build incremental append & dedupe hourly. I have noticed that our table actually does have deletes, which my current strategy cannot handle.
Our dev team is unable to add a soft delete flag to these records for the next few months and has shut down the idea of enabling CDC onto the database.
Does anyone have any ideas of how to keep these tables set to build incremental append & deduped while also set up some sort of automated way to fully refresh the tables at a monthly cadence?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.