Incremental Load with Composite Keys in Airbyte

Summary

User inquires about handling incremental data loads from Microsoft SQL Server to Postgres using composite primary keys, noting current limitations and seeking workarounds.


Question

Hi guys! My team and I are working with Airbyte in order to extract some tables from Microsoft SQL server to Postgres. Since we have quite some large tables, we would like to load data incrementally. However, our tables have multiple (composite) primary keys. As we understand from other discussions, this seems currently not possible out of the box for this source/destination combination. Before we modify any connections ourselves, as we do not have a way to alter the source tables, and CDC seems also not possible, we were wondering if any of you ever found a way around this composite keys for incremental loads issue? Thanks!



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['incremental-load', 'composite-keys', 'microsoft-sql-server', 'postgres', 'airbyte']