Issue with confirmed_flush_lsn, table sync, and publication setup in Airbyte for PostgreSQL


Airbyte is not moving confirmed_flush_lsn in PostgreSQL, sync does not stop when a table is empty, and publication setup seems unnecessary. Looking for insights on these issues.


Hi everybody, I’m working on validating Airbyte to replace our actual data workflow based in debezium and I’m facing some unwanted behaviours.
First of all the confirmed_flush_lsn is not moving forward on postgresql side, si the werver has to keep all the wal file, which is a blocking point for us
Second when a table is empty the first sync never stop still a line is inserted in the table (this is not a blocking point we do not have a lot of empty tables :smile: )
Third, more a question why a publication is needed to be set up if it is not used ? Airbyte see all the tables and not only the one defined in the publication
I’m new on Airbyte, but experienced in years on PostgreSQL, I may missed some points, even if looked widely on internet, I only found <Issues · airbytehq/airbyte · GitHub issue> really near to my concerns
My setup is PG16 (test also on PG15) and Airbyte 0.50.47
BTW congrats all team Airbyte looks really amazing, sure it will be central in lots of data team, it looks like the long-awaited tool :+1:

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["airbyte", "postgresql", "confirmed-flush-lsn", "table-sync", "publication-setup"]