Incremental Loading with Composite Keys from Microsoft SQL Server to Postgres

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Currently, it seems not possible out of the box to perform incremental loads with composite primary keys from Microsoft SQL Server to Postgres in Airbyte. The user is seeking advice on potential workarounds for this issue.


Question

Hi guys! My team and I are working with Airbyte in order to extract some tables from Microsoft SQL server to Postgres. Since we have quite some large tables, we would like to load data incrementally. However, our tables have multiple (composite) primary keys. As we understand from other discussions, this seems currently not possible out of the box for this source/destination combination. Before we modify any connections ourselves, as we do not have a way to alter the source tables, and CDC seems also not possible, we were wondering if any of you ever found a way around this composite keys for incremental loads issue? Thanks!


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["incremental-loading", "composite-keys", "microsoft-sql-server", "postgres", "workaround"]
2

Build a view on top of the table that uses the composite key, and then have Airbyte source from the view?

3

Thanks for the reply! Yeah we were thinking the same, but we only can get read access from the source database. I guess we will think about an alternative approach.