Troubleshooting 504 Unknown Error When Configuring Source Connector

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

User is facing a 504 Unknown error when configuring a source connector in Airbyte. They have tried updating connectors and suspect it may not be a timeout issue. Looking for advice on troubleshooting the error.


Question

Hi. I just recently came across Airbyte while searching for a simple data pipeline solution. I installed AirByte with abctl tool abctl local install on an EC2 t2.large instance following the QuickStart guide. I can access the the AirByte web console and I updated all source and destination connectors. Then I tried to configure a source but regardless which connector I choose I always get a 504 Unknown error message. Here on slack there are some messages that relate to a timeout error, but I couldn’t find where I need to increase that timeout value. However I doubt that “my” error relates to a timeout. E.g. when I try the docker marketplace source with my docker account (that only has 1 repo) i get the 504 as well.

Could anyone share some advice on how to troubleshoot the error?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["504-unknown-error", "source-connector", "troubleshoot", "timeout", "docker-marketplace"]
2

abctl uses kind (https://kind.sigs.k8s.io/) under the hood

install kubectl (https://kubernetes.io/docs/tasks/tools/#kubectl) and kubectx + kubens (https://github.com/ahmetb/kubectx)

KUBECONFIG=~/.airbyte/abctl/abctl.kubeconfig kubectl config view --flatten > ~/.kube/config
kubectx kind-airbyte-abctl
kubens airbyte-abctl```
then you can use `kubectl` as with regular Kubernetes cluster

you can also install k9s (<https://k9scli.io/>)
handy tool for e.g. checking what is happening with pods in cluster
3

When it comes to timeouts, I have something like this in my values.yaml file

  env_vars:
    HTTP_IDLE_TIMEOUT: 1800s

webapp:
  ingress:
    enabled: true
    annotations:
      <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: internal
      <http://nginx.ingress.kubernetes.io/proxy-body-size|nginx.ingress.kubernetes.io/proxy-body-size>: 16m
      <http://nginx.ingress.kubernetes.io/proxy-send-timeout|nginx.ingress.kubernetes.io/proxy-send-timeout>: 1800
      <http://nginx.ingress.kubernetes.io/proxy-read-timeout|nginx.ingress.kubernetes.io/proxy-read-timeout>: 1800```
you can pass values `abctl local install --values values.yaml`