Issue with custom connector incremental sync pulling duplicate rows

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

Custom connector built via Builder on oss is pulling duplicate rows for records from 2024 onwards. User is seeking help to confirm if it’s a configuration issue.


Question

I’m running into an issue with a custom connector built via Builder on oss. I have an incremental sync pulling records on YearWeek as far back as 2017. The issue I’m having is that the data is fine 2017 to 2023. 202401 until now, I get duplicate rows. Can anyone help me confirm it’s not a configuration issue? Would I just post yaml here?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["custom-connector", "incremental-sync", "duplicate-rows", "builder", "configuration-issue"]
2

What type of field is the cursor built on (e.g. creation date, last updated, etc.), and what type of field is being used for the primary key(s)? Also, can you confirm that you’re using the Incremental Sync - Append + Deduped sync type on the stream in the actual connection?