AlloyDB Postgres Connector Synchronization Failure

Summary

User is experiencing synchronization failures with the Postgres connector for AlloyDB using CDC, receiving an error related to replication slot’s confirmed LSN. They have adjusted settings but the issue persists.


Question

Hi,
I’m trying to set up data extraction from AlloyDB using the Postgres connector version 3.6.22 on Airbyte v1.1, utilizing CDC. However, the synchronization keeps failing with the following error:
“Saved offset is before replication slot’s confirmed LSN. Please reset the connection, and then increase WAL retention and/or increase sync frequency to prevent this from happening in the future.”
The settings on the AlloyDB side appear to be correct:
max_slot_wal_keep_size = 800 GB
wal_keep_size = 100 GB.
I’ve already reduced the sync frequency to every 3 hours, but the synchronization still fails. The AlloyDB instance is relatively small, and I’m only trying to synchronize a single table using WAL logs at this point.
Has anyone encountered a similar issue? Do you have any suggestions for resolving this problem?

Any advice or guidance would be greatly appreciated.



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['alloydb', 'postgres-connector', 'cdc', 'synchronization-failure', 'replication-slot', 'wal-retention']

Hello Ferdinand, have you opened an issue? I’m going to ask the database team to take a look. Please share the issue here. Do you mind sharing also what destination are you using?

Hello, sorry, the destination used is BigQuery (v2.9.0).
https://github.com/airbytehq/airbyte/issues/48744