Summary
User is facing timeout issue when trying to create a connection using Terraform provider in Airbyte installed via Helm/Docker. The error message indicates a 504 Gateway Time-out, and the user suspects it might be related to an NGINX timeout limit that needs to be increased.
Question
This is driving me nuts - I have:
Airbyte 1.0 installed via Helm/Docker
INFO Using Kubernetes provider:
Provider: kind
Kubeconfig: /Users/Jason.Maddern/.airbyte/abctl/abctl.kubeconfig
Context: kind-airbyte-abctl
SUCCESS Found Docker installation: version 27.1.1
SUCCESS Existing cluster 'airbyte-abctl' found
INFO Found helm chart 'airbyte-abctl'
Status: deployed
Chart Version: 1.0.0
App Version: 1.0.0
INFO Found helm chart 'ingress-nginx'
Status: deployed
Chart Version: 4.11.2
App Version: 1.11.2```
Terraform <https://github.com/airbytehq/terraform-provider-airbyte> v0.6.4
Both of those ^ were current versions when installed two days ago.
I can create terraform sources/destinations successfully. However when attempting to create a connection it times out with:
```airbyte_connection.test_apply_australia_tf: Creating...
airbyte_connection.test_apply_australia_tf: Still creating... [10s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [20s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [30s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [40s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [50s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [1m0s elapsed]
╷
│ Error: failure to invoke API
│
│ with airbyte_connection.test__tf,
│ on <http://connections.tf|connections.tf> line 5, in resource "airbyte_connection" "test__tf":
│ 5: resource "airbyte_connection" "test_tf" {
│
│ unknown status code returned: Status 504
│ <html>
│ <head><title>504 Gateway Time-out</title></head>
│ <body>
│ <center><h1>504 Gateway Time-out</h1></center>
│ <hr><center>nginx</center>
│ </body>
│ </html>```
I can create the same connection from the UI (so its not timing out on the fetch of the database schema). I cannot for the life of me see how I would configure this when using the `abctl` install - looks like an NGINX timeout limit to increase, I just have no idea how/where to set this. Can anyone validate if its an nginx timeout and if so how it can be set when installed through `acbtl` as it doesn't look to be a value under the `-values.yaml`
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1727410638693389) if you want
to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
["airbyte", "terraform-provider", "helm", "docker", "timeout", "504-gateway-time-out", "nginx", "abctl", "values.yaml"]
</sub>
Do you have any idea which endpoint was timing out? In 0.18.0 we bumped the timeout to 10 minutes, but sounds like you needed 20 minutes. That’s an eternity in http request time. I’m not sure what’s taking so long.
What version of abctl? abctl version
We released some nginx ingress config changes in 0.18.0 this week
Ah, I’m 17. I’ll upgrade that first - thanks Alex
version: v0.18.0```
It still times out and crashes at 60 seconds <@U07FH2Y34A1>
```airbyte_connection.test_apply_australia_tf: Creating...
airbyte_connection.test_apply_australia_tf: Still creating... [10s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [20s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [30s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [40s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [50s elapsed]
airbyte_connection.test_apply_australia_tf: Still creating... [1m0s elapsed]
╷
│ Warning: Applied changes may be incomplete
│
│ The plan was created with the -target option in effect, so some changes requested in the configuration may have been ignored and the output values may not be fully
│ updated. Run the following command to verify that no other changes are pending:
│ terraform plan
│
│ Note that the -target option is not suitable for routine use, and is provided only for exceptional situations such as recovering from errors or mistakes, or when
│ Terraform specifically suggests to use it as part of an error message.
╵
╷
│ Error: failure to invoke API
│
│ with airbyte_connection.test_apply_australia_tf,
│ on <http://connections.tf|connections.tf> line 5, in resource "airbyte_connection" "test_apply_australia_tf":
│ 5: resource "airbyte_connection" "test_apply_australia_tf" {
│
│ unknown status code returned: Status 504
│ <html>
│ <head><title>504 Gateway Time-out</title></head>
│ <body>
│ <center><h1>504 Gateway Time-out</h1></center>
│ <hr><center>nginx</center>
│ </body>
│ </html>```
Any ideas how I can increase that nginx timeout? I've read that I can apply custom values through a command like `abctl local install --values ./values.yaml` but I have no idea what that value in that file needs to be?
based on https://airbytehq.slack.com/archives/C021JANJ6TY/p1726763091482779?thread_ts=1726756277.348269&cid=C021JANJ6TY
docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl annotate ingress ingress-abctl <http://nginx.ingress.kubernetes.io/proxy-send-timeout=1800|nginx.ingress.kubernetes.io/proxy-send-timeout=1800>
docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl annotate ingress ingress-abctl <http://nginx.ingress.kubernetes.io/proxy-read-timeout=1800|nginx.ingress.kubernetes.io/proxy-read-timeout=1800>
I’m rebuilding the cluster and will try again
it shouldn’t be necessary, nginx ingress usually applies configuration instantly
Name: ingress-abctl
Labels: <none>
Namespace: airbyte-abctl
Address: 10.96.117.161
Ingress Class: nginx
Default backend: <default>
Rules:
Host Path Backends
---- ---- --------
*
/ airbyte-abctl-airbyte-webapp-svc:http (10.244.0.12:8080)
Annotations: <http://nginx.ingress.kubernetes.io/proxy-read-timeout|nginx.ingress.kubernetes.io/proxy-read-timeout>: 1800
<http://nginx.ingress.kubernetes.io/proxy-send-timeout|nginx.ingress.kubernetes.io/proxy-send-timeout>: 1800```
Thank you so much - all resolved with those two commands. That had me stumped for way too long
FYI - I was rebuilding as I was getting desperate (before you provided those commands)