PostgreSQL CDC sync writing entire rows instead of incremental append+dedupe in Airbyte

Summary

When using logical replication and CDC in PostgreSQL with Airbyte configured for incremental append+dedupe sync mode, entire rows are being written during each sync. User is seeking clarification on whether this behavior is expected or if additional configuration is required in Airbyte or the data source.


Question

I’ve set up logical replication and enabled CDC in PostgreSQL, configuring the sync mode in Airbyte as incremental append+dedupe. However, during each sync, it’s writing entire rows from the table. Is this expected behavior, or is there additional configuration needed in Airbyte or the source? can any enlighten me please



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["postgresql", "cdc", "logical-replication", "airbyte", "incremental-append+dedupe"]

and when i ran this
SELECT slot_name,
database,
plugin,
slot_type,
active,
confirmed_flush_lsn,
restart_lsn,
pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained_size
FROM pg_replication_slots;
and airbyte as subscriber or whatever the user i have created for the air byte sync is not able take all changes ?

please can any one help me with this even though my data wont get chnaged still its says all the row were extracted why incremental sync is behaving like this , of logical replication using cdc, how does everytime syncs the same data again from the sync

Does any one know how to fix this ?