Summary
Issue connecting Airbyte hosted in GCP to Snowflake trial account resulting in a 502 error after timeout. Other connections to SaaS tools and databases like Chargebee and Looker are working fine.
Question
Hey guys!
We have an issue connecting Airbyte which is hosted in GCP in Kubernetes and Snowflake trial account in GCP as well We did all the steps provided in the airbyte guide https://docs.airbyte.com/integrations/destinations/snowflake However, once we try the connection we have 502 error, which shown after the timeout
For example connection between airbyte and other SaaS tool (Chargebee) to pull the data from Chargebee works as expected
And we were able to connect Snowflake and Looker
Does anyone have ideas what can be wrong?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["502-error", "airbyte", "gcp", "kubernetes", "snowflake", "saas", "chargebee", "looker"]
can you provide logs from pods? versions of Airbyte and connectors?
HTTP 502 is very generic error without further details
Also, when you say “once we try the connection” is the the connection check on the source, or actually running a sync?
Connection checking is a common source of timeouts on k8s because it often needs to spin up a node (which takes a bit) to run that job. When an actual sync is running, this usually isn’t an issue since it’s happening in the background (so you don’t get a client timeout).
On GCP, where you need to set timeouts depends a lot on your deployment. If you’re sitting behind a Load Balancer, make sure you increase the timeout on the backend settings there, since it isn’t aware of Airbyte’s configured settings. Same goes if you’re configuring another ingress, such as your own nginx proxy.
We’re using a GKE Autopilot cluster, and bumping on the HTTP/S LB backend worked fine for us when we first set things up and haven’t had an issue since.
snowflake version is 8.36.2
it feels like java.io.UncheckedIOException: java.net.SocketException: Socket closed and Swallowed error need further investigation, but they’re a bit cryptic for me
thank you <@U05JENRCF7C>!
we tried to increase timeouts but it didn’t work as well