Custom DBT Transform Failing

Connector Questions & Issues
,
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu
  • Deployment: K8s via Helm
  • Airbyte Version: * 0.35.12-alpha
  • Step: Custom Transform
  • Description:Custom DBT transform seems to connect, pull down the repo, but fails when running the command.

See last line of the uploaded logs.

2022-06-29 19:09:12 e[42mnormalizatione[0m > Cloning into 'git_repo'...
2022-06-29 19:09:12 e[42mnormalizatione[0m > Last 5 commits in git_repo:
2022-06-29 19:09:12 e[42mnormalizatione[0m > 244ec6d test dbt
2022-06-29 19:09:12 e[42mnormalizatione[0m > 2cc3594 Remove
2022-06-29 19:09:12 e[42mnormalizatione[0m > 9d27102 commit
2022-06-29 19:09:12 e[42mnormalizatione[0m > 640f2da Test commit
2022-06-29 19:09:12 e[42mnormalizatione[0m > d79f103 Add LICENSE
2022-06-29 19:09:12 e[42mnormalizatione[0m > /config
2022-06-29 19:09:12 e[42mnormalizatione[0m > Running: transform-config --config destination_config.json --integration-type postgres --out /config
2022-06-29 19:09:12 e[42mnormalizatione[0m > Namespace(config='destination_config.json', integration_type=<DestinationType.postgres: 'postgres'>, out='/config')
2022-06-29 19:09:12 e[42mnormalizatione[0m > transform_postgres
2022-06-29 19:09:43 e[45mdbte[0m > entrypoint.sh: line 6: cd: git_repo: No such file or directory

logs-412.txt (132.8 KB)

2

I also ran into this problem a few days ago.
This is a known problem. Custom dbt transformation is not supported for kubernetes deploys Airbyte. It is stated in the Airbyte kubernetes document: https://docs.airbyte.com/deploying-airbyte/on-kubernetes/
And the issue is open in this ticket: Fix Operations API on Kube: Refactor normalization transform_config into Java · Issue #5091 · airbytehq/airbyte · GitHub

3

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.