Summary
User faces an issue with the Postgres Source using CDC for replication from an AWS Aurora database, encountering frequent errors related to WAL segment retention and replication slots.
Question
I am running into an issue with the Postgres Source using the CDC update method to replicate changes from an AWS Aurora Postgres database. Even though I have the db has very low transaction pressure, and I have configured it to retain WAL segments until they are consumed, the connector keeps failing with the following error:
> Saved offset is before replication slot’s confirmed lsn. Please reset the connection, and then increase WAL retention and/or increase sync frequency to prevent this from happening in the future. See https://docs.airbyte.com/integrations/sources/postgres/postgres-troubleshooting#under-cdc-incremental-mode-there-are-still-full-refresh-syncs for more details.
No matter how frequently I set the syncs, this occurs at least weekly, and I have to delete and recreate the source + connector to fix it. Has anyone else run into this?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.