Understanding WAL growth despite clean logs and successful data replication in Airbyte

Summary

Explanation needed on why Write-Ahead Logging (WAL) continues to grow in Postgres despite successful data replication from Postgres to Snowflake via Airbyte with no errors in logs and all streams processed.


Question

Really hoping someone can help me here :slightly_smiling_face:

Can anyone explain how the WAL continues to grow despite Airbyte having a clean log and success?

Source: Postgres (running in AWS)
Destination: Snowflake
Tables: There are only 3 tables in the database publication/replication slot

There are no errors in the logs, all streams processed. I simply don’t get it



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["wal-growth", "postgres", "aws", "snowflake", "data-replication", "airbyte"]