Is there a way to configure Airbyte so it does not unnest repeated records in separate tables for Shopify? I already have the code to do it and it would avoid the proliferation of tables. Thanks!
- Is this your first time deploying Airbyte?: Yes
- OS Version / Instance: Debian Linux 10
- Memory / Disk:
- 8 vCPUs (GCP c2-standard-8 - Intel Cascade Lake)
- 32 Gib RAM
- disk: 100 + 200 GB
- Deployment: Docker
- Airbyte Version: 0.35.5-alpha
- Shopify source: 0.1.36
- Bigquery destination: 0.6.12
- Step: Setting up a new Shopify import to BigQuery
Today you need to have a custom dbt for that @louism. You can export the dbt normalization project and remove all nested tables from there only keeping the high level ones.
I created an issue in Github Normalization: unnest only first level · Issue #11979 · airbytehq/airbyte · GitHub to implement this feature.
Thank you. Can you point me to some documentation as to where those normalizations reside and how to export them please?
I strongly recommend to you read the 3 parts tutorial about normalization: