Postgres Source CDC Update Method Issue

Summary

User faces an issue with the Postgres Source using CDC for replication from an AWS Aurora database, encountering frequent errors related to WAL segment retention and replication slots.


Question

I am running into an issue with the Postgres Source using the CDC update method to replicate changes from an AWS Aurora Postgres database. Even though I have the db has very low transaction pressure, and I have configured it to retain WAL segments until they are consumed, the connector keeps failing with the following error:
> Saved offset is before replication slot’s confirmed lsn. Please reset the connection, and then increase WAL retention and/or increase sync frequency to prevent this from happening in the future. See https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting#under-cdc-incremental-mode-there-are-still-full-refresh-syncs for more details.
No matter how frequently I set the syncs, this occurs at least weekly, and I have to delete and recreate the source + connector to fix it. Has anyone else run into this?



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['postgres-source', 'cdc-update', 'aws-aurora', 'wal-segment', 'replication-slot', 'error']