- Is this your first time deploying Airbyte?: No
- OS Version / Instance: GCP
- Memory / Disk: 16Gb
- Deployment: Docker
- Airbyte Version: 0.39.32-alpha
- Source name/version: Postgres 0.4.31
- Destination name/version: Bigquery 1.1.9
- Step: Sync job (normalization)
- Description:
Logical replication fails, with error message such as -
2022-07-11 15:48:08 normalization > 15:48:03.291819 [info ] [MainThread]: Completed with 1 error and 0 warnings:
2022-07-11 15:48:08 normalization > 15:48:03.292511 [info ] [MainThread]:
2022-07-11 15:48:08 normalization > 15:48:03.293065 [error] [MainThread]: Database Error in model transactions_in_scd (models/generated/airbyte_incremental/scd/raw_achilles/transactions_in_scd.sql)
2022-07-11 15:48:08 normalization > 15:48:03.293622 [error] [MainThread]: Invalid timestamp string "0000-12-30T00:00:00Z"
2022-07-11 15:48:08 normalization > 15:48:03.294238 [error] [MainThread]: compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_incremental/scd/raw_achilles/transactions_in_scd.sql
Logs -
logs-89696.txt (1.6 MB)
source_catalog.json
source_catalog.json.txt (7.4 KB)
The issue being with transactions_in.settlement_date
Postgres type of the column - timestamp with time zone
Like the OP, the timestamp in the data is actually 0001-01-01T00:00:00Z, which perhaps gets converted to 0000-12-30T00:00:00Z causing it to error out (as per error logs)
This is a repost of this topic, to mitigate the backend ticket process issue with discourse.