Summary
Facing statement timeout during normalization step for a large connector (50Go table) when migrating to Postgres Destination V2. Even after increasing timeout to 6 hours, still encountering failures. Replication set up with Incremental + Append mode. Normalization process should be a split of the
_airbyte_datafield, which normally takes less than 30 minutes when implemented manually.
Question
Hello, I’m trying to migrate to Postgres Destination V2, but am facing issues with the initial sync on one large connector (50Go table) : the normalisation step fails with statement timeout. I’ve tried increasing it up to 6h but still get failures. We have set up the replication with Incremental + Append mode. This seems wrong to me that it takes so long as the normalization should simply be a split of the _airbyte_data field, which our database can absorb in <30min when implemented myself.
Anyone facing similar issues ? Thanks !
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.