My environment is as follows
- Deployment: Are you using Docker or Kubernetes deployment? Yes
- Airbyte Version: What version are you using now? v0.40.26
- Source name/version: Salesforce/1.0.30 & TikTok Marketing/2.0.1
- Destination name/version: BigQuery/1.2.13
- Step: The issue is happening during sync, creating the connection or a new source? During sync
This problem has been occurring since September 2022 and I assume it is a bug in the update.
These bugs are happening in the normalization procedure, which is very costly to manage as you have to create your own json parsing process and dbt.
(_airbyte_raw_*** has the right data)
Another problem is that it is no longer possible to use sync mode for Deduped History.
Is there any movement to try to fix these?
If not, I would like to attempt to fix them myself. Can you describe the bug in detail?
@marcosmarxm
@tuliren
You guys have several actions to take on this issue. Do you have any ideas for solutions?
Related Links
- Issue
- Re-evaluate Avro logical type union · Issue #17011 · airbytehq/airbyte · GitHub
- Destination BigQuery throws error Invalid date-time: No timezone information: · Issue #17319 · airbytehq/airbyte · GitHub
- Destination S3: date type written as dictionary for parquet format · Issue #14028 · airbytehq/airbyte · GitHub
- Trouble shoot
- Slack Thread