- Is this your first time deploying Airbyte?: Yes
- OS Version / Instance: MacOS
- Memory / Disk: you can use something like 16Gb
- Deployment: docker
- Airbyte Version: 3.7
- Source name/version: appstore
- Destination name/version: bigquery
- Step: The issue is happening creating the connection
- Description: I have set up a connection between appstore and bigquery. It succesfully sync the schemas for
- subscription_event_report
- subscription_report
but fails for
- sales_report with the attached error message
logs-12.txt (1.3 MB)
I have the following settings for replication and transformation
Hey, It looks like the format of date is not in the expected format for bigquery thus causing the issue.
- You could try with a different destination otherwise
- You can use this API https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#post-/v1/connections/update to update the sync catalog and change that column to string
- Later maybe use Custom DBT to change it into date type
Hi Harshith
Thank you for your quick answer.
For my current problem I think solution 2 would be a good solution, and agreed down the line Custom DBT might be relevant.
However, looking at the existing catalog file that is being used: airbyte/configured_catalog.json at master · airbytehq/airbyte · GitHub it seems like it is already all strings, so I don’t think anything would change if I post the configured_catalog to the endpoint.
Or is it something else that I have to send to the endpoint that you just shared?
Hey could you check by hitting the API the schema and probably share it here?
Hi harshith, regize.
I am also encountering the same problem.
Apparently the date format I get from the App Store is USA (DD/MM/YYYYY).
BigQuery does not support this date format.
Is there any good way to control this problem?
