Summary
Need a solution to capture deleted records from Snowflake source to Postgres destination using Airbyte’s incremental sync, as CDC is not supported with Snowflake as a source.
Question
Hello everyone,
I am Ayushi Haria, working as a data engineer. Currently I am working on a task wherein I am copying the data from Snowflake and pasting it to the Postgres, via Airbyte. So basically
Source: Snowflake
Destination: Postgres
Sync: Incremental Deduped History
Now, the challenge is, there are few records that are getting deleted from the source, and I want that to be reflected in the destination, that can be achieved via CDC (Change Data Capture), but I came across the information that states CDC is not supported when Snowflake is source. Is anyone aware about any solution to get the information about the deleted records in the destination with the incremental sync? Can you please help me get the solution to this challenge?
I have also raised a ticket for the same: https://airbytehq.slack.com/archives/C021JANJ6TY/p1681941353967729
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.