Summary
The Airbyte Postgres connector is not correctly handling full-refresh/overwrite streams, as it fetches new records since the last sync while also overwriting existing data, resulting in only new records being left in the streams. Resetting the data temporarily resolves the issue, but it fails again in the next sync.
Question
Hi. Does anyone have problems with the Airbyte Postgres connector? I have noumerous streams syncing, some as full-refresh/overwrite and some as incremental append. However, the full-frefresh overwrite streams seems to also only fetch new records since the last sync while also overwriting. So these streams are left with only the new records. I have reset the data, and then it emitts all data, but then the next sync fails agian. Thanks
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.