Missing rows during initial incremental refresh from Postgres to Snowflake

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Issues with missing rows during initial incremental refresh from Postgres to Snowflake using Incremental (Append + Deduped) option. Error in logs related to schema validation errors for a specific stream.


Question

Hi, Has anyone faced issues with missing rows during initial incremental refresh from source(Postgres) to target(Snowflake)? I am using the option of Incremental (Append + Deduped). We are copying close to 600k records and we can see around 60 records missing from the target table after the sync is completed. Found this error in the logs “2024-06-25 18:00:26 WARN i.a.w.i.FieldSelector(lambda$reportMetrics$1):124 - Schema validation errors found for stream <table_name>. Error messages: [$.created_date: 2024-06-25T17:00:27.462250 is an invalid date-time,” Any feedback would be greatly appreciated.


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["missing-rows", "initial-incremental-refresh", "postgres", "snowflake", "incremental-append-deduped", "schema-validation-errors"]