Handling at-least-once concept in Airbyte without code


Exploring how to handle the at-least-once concept in Airbyte without the need for code and discussing deduplication on the destination side.


Hello everyone,
I started trialing airbyte locally, and I stumbled upon the updates with regards to the at-least-once concept, which means duplicated last records each time you sync (greater than or equal cursor). How are people handling this without the need for code? Do you end-up deduplicating on the destination side, outside of the scope of airbyte? I tried incremental&dedup as well with the same results
Thanks in advance

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["at-least-once-concept", "deduplication", "incremental-sync", "airbyte-platform"]