Trouble creating connection with Terraform provider in Airbyte installed via Helm/Docker

Summary

User is facing timeout issue when trying to create a connection using Terraform provider in Airbyte installed via Helm/Docker. The error message indicates a 504 Gateway Time-out, and the user suspects it might be related to an NGINX timeout limit that needs to be increased.


Question

This is driving me nuts - I have:

Airbyte 1.0 installed via Helm/Docker

 INFO   Using Kubernetes provider:
       Provider: kind
       Kubeconfig: /Users/Jason.Maddern/.airbyte/abctl/abctl.kubeconfig
       Context: kind-airbyte-abctl
 SUCCESS  Found Docker installation: version 27.1.1                                                 
 SUCCESS  Existing cluster 'airbyte-abctl' found                                                   
 INFO   Found helm chart 'airbyte-abctl'                                                      
       Status: deployed
       Chart Version: 1.0.0
       App Version: 1.0.0
 INFO   Found helm chart 'ingress-nginx'                                                      
       Status: deployed
       Chart Version: 4.11.2
       App Version: 1.11.2```
Terraform <https://github.com/airbytehq/terraform-provider-airbyte> v0.6.4

Both of those ^ were current versions when installed two days ago.

I can create terraform sources/destinations successfully. However when attempting to create a connection it times out with:

```airbyte_connection.test_apply_australia_tf: Creating...
airbyte_connection.test_apply_australia_tf: Still creating... [10s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [20s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [30s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [40s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [50s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [1m0s elapsed]
╷
│ Error: failure to invoke API
│ 
│   with airbyte_connection.test__tf,
│   on <http://connections.tf|connections.tf> line 5, in resource "airbyte_connection" "test__tf":
│    5: resource "airbyte_connection" "test_tf" {
│ 
│ unknown status code returned: Status 504
│ &lt;html&gt;
│ &lt;head&gt;&lt;title&gt;504 Gateway Time-out&lt;/title&gt;&lt;/head&gt;
│ &lt;body&gt;
│ &lt;center&gt;&lt;h1&gt;504 Gateway Time-out&lt;/h1&gt;&lt;/center&gt;
│ &lt;hr&gt;&lt;center&gt;nginx&lt;/center&gt;
│ &lt;/body&gt;
│ &lt;/html&gt;```
I can create the same connection from the UI (so its not timing out on the fetch of the database schema). I cannot for the life of me see how I would configure this when using the `abctl` install - looks like an NGINX timeout limit to increase, I just have no idea how/where to set this. Can anyone validate if its an nginx timeout and if so how it can be set when installed through `acbtl` as it doesn't look to be a value under the `-values.yaml`

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1727410638693389) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["airbyte", "terraform-provider", "helm", "docker", "timeout", "504-gateway-time-out", "nginx", "abctl", "values.yaml"]
</sub>

Do you have any idea which endpoint was timing out? In 0.18.0 we bumped the timeout to 10 minutes, but sounds like you needed 20 minutes. That’s an eternity in http request time. I’m not sure what’s taking so long.

What version of abctl? abctl version

We released some nginx ingress config changes in 0.18.0 this week

Ah, I’m 17. I’ll upgrade that first - thanks Alex

version: v0.18.0```
It still times out and crashes at 60 seconds <@U07FH2Y34A1>

```airbyte_connection.test_apply_australia_tf: Creating...
airbyte_connection.test_apply_australia_tf: Still creating... [10s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [20s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [30s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [40s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [50s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [1m0s elapsed]
╷
│ Warning: Applied changes may be incomplete
│ 
│ The plan was created with the -target option in effect, so some changes requested in the configuration may have been ignored and the output values may not be fully
│ updated. Run the following command to verify that no other changes are pending:
│     terraform plan
│ 
│ Note that the -target option is not suitable for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when
│ Terraform specifically suggests to use it as part of an error message.
╵
╷
│ Error: failure to invoke API
│ 
│   with airbyte_connection.test_apply_australia_tf,
│   on <http://connections.tf|connections.tf> line 5, in resource "airbyte_connection" "test_apply_australia_tf":
│    5: resource "airbyte_connection" "test_apply_australia_tf" {
│ 
│ unknown status code returned: Status 504
│ &lt;html&gt;
│ &lt;head&gt;&lt;title&gt;504 Gateway Time-out&lt;/title&gt;&lt;/head&gt;
│ &lt;body&gt;
│ &lt;center&gt;&lt;h1&gt;504 Gateway Time-out&lt;/h1&gt;&lt;/center&gt;
│ &lt;hr&gt;&lt;center&gt;nginx&lt;/center&gt;
│ &lt;/body&gt;
│ &lt;/html&gt;```
Any ideas how I can increase that nginx timeout? I've read that I can apply custom values through a command like `abctl local install --values ./values.yaml` but I have no idea what that value in that file needs to be?

based on https://airbytehq.slack.com/archives/C021JANJ6TY/p1726763091482779?thread_ts=1726756277.348269&amp;cid=C021JANJ6TY

docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl annotate ingress ingress-abctl <http://nginx.ingress.kubernetes.io/proxy-send-timeout=1800|nginx.ingress.kubernetes.io/proxy-send-timeout=1800>
docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl annotate ingress ingress-abctl <http://nginx.ingress.kubernetes.io/proxy-read-timeout=1800|nginx.ingress.kubernetes.io/proxy-read-timeout=1800>

I’m rebuilding the cluster and will try again :slightly_smiling_face:

it shouldn’t be necessary, nginx ingress usually applies configuration instantly

Name:             ingress-abctl
Labels:           &lt;none&gt;
Namespace:        airbyte-abctl
Address:          10.96.117.161
Ingress Class:    nginx
Default backend:  &lt;default&gt;
Rules:
  Host        Path  Backends
  ----        ----  --------
  *
              /   airbyte-abctl-airbyte-webapp-svc:http (10.244.0.12:8080)
Annotations:  <http://nginx.ingress.kubernetes.io/proxy-read-timeout|nginx.ingress.kubernetes.io/proxy-read-timeout>: 1800
              <http://nginx.ingress.kubernetes.io/proxy-send-timeout|nginx.ingress.kubernetes.io/proxy-send-timeout>: 1800```

Thank you so much - all resolved with those two commands. That had me stumped for way too long

FYI - I was rebuilding as I was getting desperate (before you provided those commands) :melting_face: