Normalization failed during AppStore dbt run for the sales_report

  • Is this your first time deploying Airbyte?: Yes
  • OS Version / Instance: MacOS
  • Memory / Disk: you can use something like 16Gb
  • Deployment: docker
  • Airbyte Version: 3.7
  • Source name/version: appstore
  • Destination name/version: bigquery
  • Step: The issue is happening creating the connection
  • Description: I have set up a connection between appstore and bigquery. It succesfully sync the schemas for
    • subscription_event_report
    • subscription_report
      but fails for
    • sales_report with the attached error message
      logs-12.txt (1.3 MB)

I have the following settings for replication and transformation

Hey, It looks like the format of date is not in the expected format for bigquery thus causing the issue.

  1. You could try with a different destination otherwise
  2. You can use this API to update the sync catalog and change that column to string
  3. Later maybe use Custom DBT to change it into date type

Hi Harshith

Thank you for your quick answer.

For my current problem I think solution 2 would be a good solution, and agreed down the line Custom DBT might be relevant.

However, looking at the existing catalog file that is being used: airbyte/configured_catalog.json at master · airbytehq/airbyte · GitHub it seems like it is already all strings, so I don’t think anything would change if I post the configured_catalog to the endpoint.

Or is it something else that I have to send to the endpoint that you just shared?

Hey could you check by hitting the API the schema and probably share it here?

Hi harshith, regize.

I am also encountering the same problem.
Apparently the date format I get from the App Store is USA (DD/MM/YYYYY).
BigQuery does not support this date format.

スクリーンショット 2022-12-09 11.09.16

Is there any good way to control this problem?