Getting 504 Gateway-time out error for Airbyte connection

ServerErrorLogs.txt (10.0 KB)

  • Is this your first time deploying Airbyte: No
  • OS Version / Instance: Ubuntu 18.04, EC2 t3.xlarge
  • Memory / Disk: 16Gb / 40GB SSD
  • Deployment: Kubernetes
  • Airbyte Version: 0.32.0-alpha
  • Source name/version: Postgres 0.3.7
  • Destination name/version: S3 0.1.13
  • Step: Setting new connection, source / On sync
  • Description: I’m trying to sync the data from Postgres to S3 and the process doesn’t finish and I start to get the attached error in server logs. Also the airbyte UI does not show the logs on UI for the same and connection job does not finish.

Hi @hokam,
I think there is a problem with the health of your deployment.
Could you please upgrade to the latest Airbyte version and redeploy Airbyte? And use Postgres connector version 0.4.4?
I have additional questions that could help us troubleshoot this:

  • How did you deploy Airbyte on Kubernetes, with our Helm Chart or Kustomize?
  • How many nodes do you have on your cluster?
  • Could please check if what does the memory consumption looks like when running this sync?

Hi @alafanechere,

Appreciate your help.

Please fine additional details that you need:

  • How did you deploy Airbyte on Kubernetes, with our Helm Chart or Kustomize?
    With Helm Chart.
  • How many nodes do you have on your cluster?
    Minimum 3 nodes and max 10 nodes of t3.xlarge.
  • Could please check if what does the memory consumption looks like when running this sync?
    Checked the memory footprint on nodes, It is reaching 30 % when this job runs.

I will try by changing the Postgres version to 0.4.4 and see if that helps.

Thanks

Could you please try to run all the Airbyte related pods in the same node? There is maybe a problem in sharing the volume data across nodes that could lead to failure in log retrieval.
Before doing this do you mind trying to set up another simple connection (e.g PokeAPI to S3) to check if the problem with the logs is a global one?

Let me know if the Postgres update makes it work better.