Issue with date column in postgres source CDC to bigquery

Platform, Deploy & Infra Issues
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: debian 10 buster
  • Memory / Disk: 8Gb / 50 GB
  • Deployment: Docker
  • Airbyte Version: v0.39.34-alpha
  • Step: The issue appears when a source postgres table has a date format and syncing with incremental + dedupe. It starts with the following error:
    2022-07-07 18:51:17 e[1;31mERRORe[m c.n.s.DateTimeValidator(isLegalDateTime):91 - The supplied date/time format type does not match the specification, expected: date

And then fails on normalization with the following errors.

- Description: Process fails with the following error messages. 
> 18:56:47.339528 [debug] [Thread-1  ]: BigQuery adapter: Retry attempt 1 of 3 after error: BadRequest("Invalid date: '2020-01-02T00:00:00Z'")
2022-07-07 18:57:46 e[42mnormalizatione[0m > 18:57:46  Database Error in model condition_scd (models/generated/airbyte_incremental/scd/public/condition_scd.sql)
2022-07-07 18:57:46 e[42mnormalizatione[0m > 18:57:46    Invalid date: '2020-01-02T00:00:00Z'
2

Hello @Matt_Hardner, welcome to the Airbyte community!

Just to double check that I understand this correctly: the source connector expects a date, but Postgres returns a datetime instead? I see a few similar issues with other connectors, and in one someone suggests a hot fix. Take a look at this comment and see if it could serve as a temporary solution while I look further into this!