All Destinations: _scd tables not empty after reset

[andmo]

• Is this your first time deploying Airbyte?: No
• OS Version / Instance: Ubuntu
• Memory / Disk: JOB_MAIN_CONTAINER_MEMORY_REQUEST=1Gi, JOB_MAIN_CONTAINER_MEMORY_LIMIT=1Gi
• Deployment: Kubernetes v1.22.9
• Airbyte Version: v0.39.17-alpha
• Source name/version: Postgres v0.4.18
• Destination name/version: Snowflake v0.4.30
• Step: Reset
• Description: I have a connection with five tables, all incremental + deduped history. I needed to change the key for tracking changes so I reset the data and re-synced. Then I was alerted by my team about duplicates in the data of these tables. I noticed that after running a reset, the _SCD tables were not being emptied and continued to have the old data. I truncated the _SCD tables after a reset and then duplicates did not appear.

I have attached an image of an _SCD table still having rows after running a reset.

[marcosmarxm]

Hello Andrew sorry the long delay tor reply here!
I created the issue https://github.com/airbytehq/airbyte/issues/14747 in Github to solve the problem.
When this is solved I’ll return to you here.

[isaacharrisholt]

Just to bump this, it’s also happening with the BigQuery destination. I’ve created an issue here.

[marcosmarxm]

Isaac, the first issue is https://github.com/airbytehq/airbyte/issues/5417. I raised the issue to cnnector team.