Error message when transferring data from Postgres replica to S3 bucket with CDC

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Error message ‘Saved offset is before replication slot’s confirmed lsn’ encountered after multiple executions with a delay. WAL files set to unlimited. Seeking help to resolve the issue.


Question

Hi , I set Airbyte to transfer data from Postgres 16 replica to S3 bucket with CDC. The 1st sync work ok but after 2-3 executions (3-5 mins pass between the execution) I’m getting the following error:

"internalMessage" : "io.airbyte.commons.exceptions.ConfigErrorException: Saved offset is before replication slot's confirmed lsn. Please reset the connection, and then increase WAL retention and/or increase sync frequency to prevent this from happening in the future
I set the WAL files to be unlimited , so it not look like its WAL files missing is the issue , Is anyone encounter this issue?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["postgres-replica", "s3-bucket", "cdc", "wal-files", "error", "replication-slot"]
2

What values do you have for max_slot_wal_keep_size, max_wal_size, wal_keep_size ?

3

max_slot_wal_keep_size = -1
max_wal_size - 2GB
wal_keep_size - 400G

4

Hmm, configuration looks good. Have you searched Slack or Github issues?
I think there were some issues like this.

5

No, not yet , i will