CDC sync Invalid date: '2016-04-15T00:00:00Z'

Connector Questions & Issues
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Amazon Linux, t2.large
  • Memory / Disk: 8,0GB / 50 GB
  • Deployment: Docker
  • Airbyte Version: 0.39.37-alpha
  • Source name/version: 0.4.36
  • Destination name/version: 1.1.11
  • Step: The issue is happening during normalization sync.
  • Description: I made a sync and all works fine, but normalization is fail. only receving this message:

2022-07-22 16:10:45 normalization > 16:10:39.520541 [info ] [MainThread]: Completed with 3 errors and 0 warnings:
2022-07-22 16:10:45 normalization > 16:10:39.521105 [info ] [MainThread]:
2022-07-22 16:10:45 normalization > 16:10:39.521525 [error] [MainThread]: Database Error in model buyers_scd (models/generated/airbyte_incremental/scd/database_cdc/buyers_scd.sql)
2022-07-22 16:10:45 normalization > 16:10:39.521984 [error] [MainThread]: Invalid date: ‘2016-04-15T00:00:00Z’
2022-07-22 16:10:45 normalization > 16:10:39.522389 [info ] [MainThread]:
2022-07-22 16:10:45 normalization > 16:10:39.522774 [error] [MainThread]: Database Error in model orders_scd (models/generated/airbyte_incremental/scd/database_cdc/orders_scd.sql)
2022-07-22 16:10:45 normalization > 16:10:39.523044 [error] [MainThread]: Invalid date: ‘2021-06-01T00:00:00Z’
2022-07-22 16:10:45 normalization > 16:10:39.523509 [error] [MainThread]: compiled SQL at …/build/run/airbyte_utils/models/generated/airbyte_incremental/scd/database_cdc/orders_scd.sql
2022-07-22 16:10:45 normalization > 16:10:39.523913 [info ] [MainThread]:
2022-07-22 16:10:45 normalization > 16:10:39.524327 [error] [MainThread]: Database Error in model payment_installments_scd (models/generated/airbyte_incremental/scd/database_cdc/payment_installments_scd.sql)
2022-07-22 16:10:45 normalization > 16:10:39.524772 [error] [MainThread]: Invalid date: ‘2022-04-28T00:00:00Z’

I made a conection without cdc and works fine. I realy don’t know what coud be.

Could you help me?

2

Hey could you help with the information on which source you are using and also share the full logs?

3

Hi @harshith , sorry don’t see your message. I did a update of airbytes and create new connection with last versions of source but error persist.

  • 0.39.39-alpha
    Source: 0.4.37

Follow log:
logs-54.txt (4.1 MB)

4

I discovery that a collum type happen. Timestamp to date. Maybe could be the normalization problem? Could’t Be?

5

Hey yeah looks like a normalisation issue. in the tables buyers, orders and payment_installation there is a data parameter which is causing this.

As a temporary fix you can update the catalog and change these to string for which

  1. https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#post-/v1/connections/get you can get the connection details in which you will find syncCatalog
  2. Change the type for those date keys to string
  3. Hit the update API and try running the sync again
6

Hi @harshith, thanks for support its working!