Issues with Airbyte Terraform Provider

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

User reports a bug with the airbyte_connection resource in the Airbyte Terraform provider, where terraform apply fails with a 400 status code due to malformed syntax after declaring a stream config, despite a successful plan.


Question

Anyone using the Airbyte Terraform provider?

I am having strange issues with the airbyte_connection resource. As soon as I declare a stream config, terraform apply starts failing with a mysterious error unknown status code returned: Status 400: The request could not be understood by the server due to malformed syntax.
Even though the Terraform plan runs without any errors.

│ stream found. The list of valid streams include: []."}}```
Full description <https://github.com/airbytehq/terraform-provider-airbyte/issues/156|here>

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1731431536708569) if you want
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
['airbyte', 'terraform', 'airbyte_connection', 'status-400', 'malformed-syntax']
</sub>
2

I made a mistake in a previous message, so I deleted it.

You can check one thing,
but first rename terraform resource postgres_to_snowflake to snowflake_to_postgres because it’s confusing.
Here are the steps:

  1. Create source and destination from terraform
  2. Create connection manually. Be sure that it detects streams properly. Import connection to terraform state like this terraform import airbyte_connection.snowflake_to_postgres "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx". Replace connection_id "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" with your value. See the screenshot where you can find it.
  3. Run terraform plan . If you find any differences, then update your terraform code.
3

Hi <@U05JENRCF7C>

I have done as you suggested. I modified the TF config to the point that it detected no changes (i.e TF config identical to actual resource). However, when I make the smallest tweak and apply, I get the same malformed syntax error

4

Can you go in UI to Schema tab in connection and press Refresh source schema, then make screenshot of existing streams?

5

I found a misconfiguration in my source

The schema I had in my airbyte_source_snowflake resource in TF had a typo. Seems to be working well now, thanks for your help! <@U05JENRCF7C>

6

Thank you for looking into this. It works fine when set up via the airbyte UI with the same user

7

I made a mistake in a previous message, so I deleted it.

You can check one thing,
but first rename terraform resource postgres_to_snowflake to snowflake_to_postgres because it’s confusing.
Here are the steps:

  1. Create source and destination from terraform
  2. Create connection manually. Be sure that it detects streams properly. Import connection to terraform state like this terraform import airbyte_connection.snowflake_to_postgres "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx". Replace connection_id "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" with your value. See the screenshot where you can find it.
  3. Run terraform plan . If you find any differences, then update your terraform code.
8

Hi <@U05JENRCF7C>

I have done as you suggested. I modified the TF config to the point that it detected no changes (i.e TF config identical to actual resource). However, when I make the smallest tweak and apply, I get the same malformed syntax error

9

Can you go in UI to Schema tab in connection and press Refresh source schema, then make screenshot of existing streams?

10

I found a misconfiguration in my source

The schema I had in my airbyte_source_snowflake resource in TF had a typo. Seems to be working well now, thanks for your help! <@U05JENRCF7C>