Summary
User reports issues with Airbyte’s integration with PostgreSQL, including problems with confirmed_flush_lsn not advancing, first sync behavior with empty tables, and questions regarding the necessity of publication setup.
Question
Hi everybody, I’m working on validating Airbyte to replace our actual data workflow based in debezium and I’m facing some unwanted behaviours.
First of all the confirmed_flush_lsn is not moving forward on postgresql side, si the werver has to keep all the wal file, which is a blocking point for us
Second when a table is empty the first sync never stop still a line is inserted in the table (this is not a blocking point we do not have a lot of empty tables )
Third, more a question why a publication is needed to be set up if it is not used ? Airbyte see all the tables and not only the one defined in the publication
I’m new on Airbyte, but experienced in years on PostgreSQL, I may missed some points, even if looked widely on internet, I only found <https://github.com/airbytehq/airbyte/issues/6230|this issue> really near to my concerns
My setup is PG16 (test also on PG15) and Airbyte 0.50.47
BTW congrats all team Airbyte looks really amazing, sure it will be central in lots of data team, it looks like the long-awaited tool
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.