Understanding WAL growth despite clean logs and successful data replication in Airbyte


Explanation needed on why Write-Ahead Logging (WAL) continues to grow in Postgres despite successful data replication from Postgres to Snowflake via Airbyte with no errors in logs and all streams processed.


Really hoping someone can help me here :slightly_smiling_face:

Can anyone explain how the WAL continues to grow despite Airbyte having a clean log and success?

Source: Postgres (running in AWS)
Destination: Snowflake
Tables: There are only 3 tables in the database publication/replication slot

There are no errors in the logs, all streams processed. I simply don’t get it

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["wal-growth", "postgres", "aws", "snowflake", "data-replication", "airbyte"]