Incremental Sync Successful but Rows in Table Duped

Connector Questions & Issues
1
  • s this your first time deploying Airbyte: No
  • OS Version / Instance: EC2 t2.large
  • Memory / Disk: 16Gb
  • Deployment: Docker
  • Airbyte Version: 0.35.56-alpha
  • Source name/version: Postgres 0.4.10
  • Destination name/version: Snowflake 0.4.24
  • Step: Sync fails
  • Description: This job replicates about 70 tables that use incremental deduped history. However, for some reason two tables did not dedupe and the job replicated identical rows. This is the first time this has happened and not sure why. A few jobs have been failing due to invalid identifiers as well before this happened despite nothing changing in the source.

logs-6464.txt (1.6 MB)

2

Hey @kyle-cheung do you see any pattern in those rows which have not deduped?

4

No pattern, i did a hard reset and dropped the tables and they resynced with no issue. But still weird that it happened!

5

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.