Issue accessing Airbyte on Google Cloud VM

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

Unable to access Airbyte via browser on Google Cloud VM, getting connection refused error


Question

I have an issue setting up airbyte on GC with a VM.

Airbyte is successfully installed. I can curl the localhost. but not reach it via the browser.

E1003 10:28:47.613520  122799 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"<https://127.0.0.1:34847/api?timeout=32s>\": dial tcp 127.0.0.1:34847: connect: connection refused"
E1003 10:28:47.615658  122799 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"<https://127.0.0.1:34847/api?timeout=32s>\": dial tcp 127.0.0.1:34847: connect: connection refused"
E1003 10:28:47.617708  122799 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"<https://127.0.0.1:34847/api?timeout=32s>\": dial tcp 127.0.0.1:34847: connect: connection refused"
E1003 10:28:47.619407  122799 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"<https://127.0.0.1:34847/api?timeout=32s>\": dial tcp 127.0.0.1:34847: connect: connection refused"
E1003 10:28:47.620960  122799 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: Get \"<https://127.0.0.1:34847/api?timeout=32s>\": dial tcp 127.0.0.1:34847: connect: connection refused"
The connection to the server 127.0.0.1:34847 was refused - did you specify the right host or port?
BM@airbyte:~$ kind get clusters
airbyte-abctl
BM:~$ kubectl config current-context
kind-airbyte-abctl
BM@airbyte:~$ kubectl --kubeconfig ~/.airbyte/abctl/abctl.kubeconfig --namespace airbyte-abctl get pods
NAME                                                      READY   STATUS      RESTARTS        AGE
airbyte-abctl-airbyte-bootloader                          0/1     Completed   0               7h55m
airbyte-abctl-connector-builder-server-77bd46c94c-226vw   1/1     Running     0               7h54m
airbyte-abctl-cron-5b5fbbcb99-68jg9                       1/1     Running     0               7h54m
airbyte-abctl-pod-sweeper-pod-sweeper-59c9f5966f-2s2xs    1/1     Running     0               7h54m
airbyte-abctl-server-7b8685fd77-qq8mk                     1/1     Running     1 (7h51m ago)   7h54m
airbyte-abctl-temporal-688489fcb6-nk8vc                   1/1     Running     0               7h54m
airbyte-abctl-webapp-58b8dfc689-bvx76                     1/1     Running     0               7h54m
airbyte-abctl-worker-695f6b4fdb-kpj9n                     1/1     Running     0               7h54m
airbyte-abctl-workload-api-server-7f6d5d677c-4j8f5        1/1     Running     0               7h54m
airbyte-abctl-workload-launcher-867db77d98-r2pzc          1/1     Running     0               7h54m
airbyte-db-0                                              1/1     Running     0               7h55m
airbyte-minio-0                                           1/1     Running     0               7h55m```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1727951764473499) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["airbyte", "google-cloud", "vm", "kubectl", "connection-refused", "browser"]
</sub>
2

what do you get for docker inspect airbyte-abctl-control-plane?

3 
    {
        "Id": "f6183ae193aae11d65af7eaab7c24a9a7deec931fca428949e77e52e7b30f747",
        "Created": "2024-10-03T02:36:10.03181204Z",
        "Path": "/usr/local/bin/entrypoint",
        "Args": [
            "/sbin/init"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 1192,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2024-10-03T11:17:00.03267529Z",
            "FinishedAt": "2024-10-03T11:14:37.760288283Z"
        },
        "Image": "sha256:529a3cc5dfdd37441b32e9916ea00daed9ada15b545c84f4a55f27d69ce197d1",
        "ResolvConfPath": "/var/lib/docker/containers/f6183ae193aae11d65af7eaab7c24a9a7deec931fca428949e77e52e7b30f747/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/f6183ae193aae11d65af7eaab7c24a9a7deec931fca428949e77e52e7b30f747/hostname",
        "HostsPath": "/var/lib/docker/containers/f6183ae193aae11d65af7eaab7c24a9a7deec931fca428949e77e52e7b30f747/hosts",
        "LogPath": "/var/lib/docker/containers/f6183ae193aae11d65af7eaab7c24a9a7deec931fca428949e77e52e7b30f747/f6183ae193aae11d65af7eaab7c24a9a7deec931fca428949e77e52e7b30f747-json.log",
        "Name": "/airbyte-abctl-control-plane",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "unconfined",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/lib/modules:/lib/modules:ro",
                "/home/bjoern_maronde/.airbyte/abctl/data:/var/local-path-provisioner"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "kind",
            "PortBindings": {
                "6443/tcp": [
                    {
                        "HostIp": "127.0.0.1",
                        "HostPort": "32923"
                    }
                ],
                "80/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "8000"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "on-failure",
                "MaximumRetryCount": 1
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "ConsoleSize": [
                0,
                0
            ],
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": true,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [
                "seccomp=unconfined",
                "apparmor=unconfined",
                "label=disable"
            ],
            "Tmpfs": {
                "/run": "",
                "/tmp": ""
            },
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": [],
            "BlkioDeviceWriteBps": [],
            "BlkioDeviceReadIOps": [],
            "BlkioDeviceWriteIOps": [],
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": [],
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": null,
            "ReadonlyPaths": null,
            "Init": false
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/5c81253c19337aa1afd5c6f68e107a78723869f2fb0cea9a0e967b2d5752a087-init/diff:/var/lib/docker/overlay2/07a3048977670325d31dd0d1a162568ae0363df630aeae3e1295fb6a3155ed9e/diff:/var/lib/docker/overlay2/1a0eec63111ad7abfaadc46ad2ff523406d347653979b8c1c1cc38485be20457/diff",
                "MergedDir": "/var/lib/docker/overlay2/5c81253c19337aa1afd5c6f68e107a78723869f2fb0cea9a0e967b2d5752a087/merged",
                "UpperDir": "/var/lib/docker/overlay2/5c81253c19337aa1afd5c6f68e107a78723869f2fb0cea9a0e967b2d5752a087/diff",
                "WorkDir": "/var/lib/docker/overlay2/5c81253c19337aa1afd5c6f68e107a78723869f2fb0cea9a0e967b2d5752a087/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/lib/modules",
                "Destination": "/lib/modules",
                "Mode": "ro",
                "RW": false,
                "Propagation": "rprivate"
            },
            {
                "Type": "volume",
                "Name": "8bc4ba82f017dd990739729a91d954338bb374a4839ffd20408977f44921d36b",
                "Source": "/var/lib/docker/volumes/8bc4ba82f017dd990739729a91d954338bb374a4839ffd20408977f44921d36b/_data",
                "Destination": "/var",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            },
            {
                "Type": "bind",
                "Source": "/home/bjoern_maronde/.airbyte/abctl/data",
                "Destination": "/var/local-path-provisioner",
                "Mode": "",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "airbyte-abctl-control-plane",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "6443/tcp": {},
                "80/tcp": {}
            },
            "Tty": true,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER",
                "KUBECONFIG=/etc/kubernetes/admin.conf",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "container=docker",
                "HTTP_PROXY=",
                "HTTPS_PROXY=",
                "NO_PROXY="
            ],
            "Cmd": null,
            "Image": "kindest/node:v1.29.8@sha256:d46b7aa29567e93b27f7531d258c372e829d7224b25e3fc6ffdefed12476d3aa",
            "Volumes": {
                "/var": {}
            },
            "WorkingDir": "/",
            "Entrypoint": [
                "/usr/local/bin/entrypoint",
                "/sbin/init"
            ],
            "OnBuild": null,
            "Labels": {
                "io.x-k8s.kind.cluster": "airbyte-abctl",
                "io.x-k8s.kind.role": "control-plane"
            },
            "StopSignal": "SIGRTMIN+3"
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "8d9fc28480a9618073694cde06ea0da04eeb2c16bd3afbc306a3452bcc361fb5",
            "SandboxKey": "/var/run/docker/netns/8d9fc28480a9",
            "Ports": {
                "6443/tcp": [
                    {
                        "HostIp": "127.0.0.1",
                        "HostPort": "32923"
                    }
                ],
                "80/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "8000"
                    }
                ]
            },
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "kind": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "MacAddress": "02:42:ac:12:00:02",
                    "DriverOpts": null,
                    "NetworkID": "e4be2dbc9cfd092df7bd4cf7ef390617a25bcfdbc68ed97bc2e6317b3530c99f",
                    "EndpointID": "1cacdf8e5adab11076062cd491385e7613f7e2552bc97b2eec804be492c88a5c",
                    "Gateway": "172.18.0.1",
                    "IPAddress": "172.18.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "fc00:f853:ccd:e793::1",
                    "GlobalIPv6Address": "fc00:f853:ccd:e793::2",
                    "GlobalIPv6PrefixLen": 64,
                    "DNSNames": [
                        "airbyte-abctl-control-plane",
                        "f6183ae193aa"
                    ]
                }
            }
        }
    }
]```
4 
    "HostIp": "0.0.0.0",
    "HostPort": "8000"
}```
entries in Ports and PortBindings look fine.

What page are trying to open in a browser, something like `http://&lt;machine-ip-address&gt;:8000`?
Do you get any error in a browser?
5

Yes - its not loading

I can curl it but not reach it via browser

6

do you have port 8000 open for this machine?
GC in the first message stands for Google Cloud?

7

yes its open.

I just try to reinstall airbyte.

But already in the installation phase I get:

            Pod: airbyte-abctl-workload-launcher-867db77d98-jsk5q.17faf1c9d6da319d
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.11:8016/health/readiness>": dial tcp 10.244.0.11:8016: connect: connection refused
            Count: 6
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-workload-launcher-867db77d98-jsk5q.17faf1c9d6da319d
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.11:8016/health/readiness>": dial tcp 10.244.0.11:8016: connect: connection refused
            Count: 7
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-server-7b8685fd77-rpdfq.17faf1ce8a44763e
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.14:8001/api/v1/health>": dial tcp 10.244.0.14:8001: connect: connection refused
            Count: 6
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-workload-launcher-867db77d98-jsk5q.17faf1c9d6da319d
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.11:8016/health/readiness>": dial tcp 10.244.0.11:8016: connect: connection refused
            Count: 8
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-workload-launcher-867db77d98-jsk5q.17faf1c9d6da319d
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.11:8016/health/readiness>": dial tcp 10.244.0.11:8016: connect: connection refused
            Count: 9
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-server-7b8685fd77-rpdfq.17faf1ce8a44763e
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.14:8001/api/v1/health>": dial tcp 10.244.0.14:8001: connect: connection refused
            Count: 7
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-workload-launcher-867db77d98-jsk5q.17faf1c9d6da319d
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.11:8016/health/readiness>": dial tcp 10.244.0.11:8016: connect: connection refused
            Count: 10
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-server-7b8685fd77-rpdfq.17faf1ce8a44763e
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.14:8001/api/v1/health>": dial tcp 10.244.0.14:8001: connect: connection refused
            Count: 8
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-workload-launcher-867db77d98-jsk5q.17faf1c9d6da319d
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.11:8016/health/readiness>": dial tcp 10.244.0.11:8016: connect: connection refused
            Count: 11
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-workload-launcher-867db77d98-jsk5q.17faf1ce7efccbc6
            Reason: Unhealthy
            Message: Liveness probe failed: Get "<http://10.244.0.11:8016/health/liveness>": dial tcp 10.244.0.11:8016: connect: connection refused
            Count: 6
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-server-7b8685fd77-rpdfq.17faf1ce8a44763e
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.14:8001/api/v1/health>": dial tcp 10.244.0.14:8001: connect: connection refused
            Count: 9
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-server-7b8685fd77-rpdfq.17faf1ce8a44763e
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.14:8001/api/v1/health>": dial tcp 10.244.0.14:8001: connect: connection refused
            Count: 10
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-server-7b8685fd77-rpdfq.17faf1ce8a44763e
            Reason: Unhealthy
            Message: Readiness probe failed: Get "<http://10.244.0.14:8001/api/v1/health>": dial tcp 10.244.0.14:8001: connect: connection refused
            Count: 11
 WARNING  Encountered an issue deploying Airbyte:                                                               
            Pod: airbyte-abctl-server-7b8685fd77-rpdfq.17faf1d3323f26cd
            Reason: Unhealthy
            Message: Liveness probe failed: Get "<http://10.244.0.14:8001/api/v1/health>": dial tcp 10.244.0.14:8001: connect: connection refused
            Count: 6
▀  Installing 'airbyte/airbyte' (version: 1.0.0) Helm Chart (this may take several minutes) (6m57s)```
8

for clean installation, I recommend abctl local uninstall --persisted first

9

another idea, you can try SSH Tunneling
https://www.ssh.com/academy/ssh/tunneling-example

10

Mhh its really tricky - could it be that ingress is causing the issues?

kind-airbyte-abctl
bjoern_maronde@airbyte:~$ kubectl config use-context kind-airbyte-abctl
Switched to context "kind-airbyte-abctl".
bjoern_maronde@airbyte:~$ kubectl logs kube-apiserver-airbyte-abctl-control-plane -n kube-system
The connection to the server 127.0.0.1:35457 was refused - did you specify the right host or port?
bjoern_maronde@airbyte:~$ kubectl get pods -n kube-system
The connection to the server 127.0.0.1:35457 was refused - did you specify the right host or port?
bjoern_maronde@airbyte:~$ ```