Dbt run return `unrecognized arguments` using --vars

Connector Questions & Issues
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu 22.04
  • Memory / Disk: 8Gb / 80 Gb NVme
  • Deployment: Docker
  • Airbyte Version: 0.40.3
  • Source name/version: Postgres/1.0.9
  • Destination name/version: Postgres/ 0.3.24
  • Step: The issue is happening during sync
  • Description:

I created a specific dbt transformation at GitHub - OpenSourcePolitics/dbt_decidim, to transform data from a postgres database to another postgres one.

At the root of my dbt, I have a variable defined in the organization.sql file (see here: dbt_decidim/organizations.sql at 7c0bd4fd7a697c1bc4d364a03082683ef3e180e8 · OpenSourcePolitics/dbt_decidim · GitHub)

...
where host = '{{ var('DBT_DECIDIM') }}'
...

When using the command dbt run --vars '{"DBT_DECIDIM": "example.com"}' locally, it works fine.

However, when adding run --vars '{"DBT_DECIDIM": "example.com"}' as entrypoint in a custom transformation on Airbyte, it returns me a dbt: error: unrecognized arguments: 'example.com')

FYI, I succeeded at reproducing the error locally when putting backslash before quotes (dbt run --vars \'{"DBT_DECIDIM": "example.com"}\'), so I think it has to deal with it, but I tried many combinations of single/double quotes and didn’t any that worked on Airbyte.

2

Please check documentation about using custom vars in normalization: https://docs.airbyte.com/operator-guides/transformation-and-normalization/transformations-with-airbyte#using-a-custom-run-with-variables

1 Like