Question about append and dedupe functionality for Stripe Disputes

Summary

The user is experiencing unexpected behavior with the append and dedupe functionality in the Stripe Disputes connector. They expected unique records based on PK and Cursor, but it seems to only keep the latest cursor record with the earliest _airbyte_extracted_at timestamp. The user is looking for a way to achieve the expected deduplication process.


Question

Question regarding append and dedupe functionality. I turned on append and deduplicate for Stripe Disputes (cursor field updated timestamp). I was expecting the functionality to keep unique PK, Cursor records to create unique event stream of changes, getting rid of duplicate records have the same PK, Cursor and keeping the one with the oldest _airbyte_extracted_at timestamp. Instead, it looks like it’s only keeping latest cursor record for a PK with the earliest _airbyte_extracted_at and getting rid of rest. Why isn’t airbyte keeping the unique record event stream, and is there a way to make it dedup like I was expecting?



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["append-dedupe", "stripe-disputes-connector", "unique-record", "deduplication", "airbyte-extracted-at"]

<@U037DKCK1MY> We didn’t talk about this one, but its another