Quick question about Incremental I Append Sync mode. In this mode, does airbyte just pull changes since last ingestion using cursor field and perform merge operation on the destination (databricks delta)? I see it only doing append is it possible to upsert instead? I am using Open source version of Airbyte
Hey if you mean to remove duplicate records then we have an option called incremental dedup which you can use
I mean something like upsert or merge into when writing to delta in databaricks.
we don’t have a upsert thing but we append and dedup. Feel free to create a ticket as improvement over github so that team can comment over it
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.