Encountering HTTP 524 error when creating a connection between BigQuery and Snowflake
Hi all. I’m encountering this error when creating a connection between BQ and Snowflake.
An unknown error occurred. (HTTP 524)
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Do you use Airbyte Cloud or Airbyte OSS?
It looks like HTTP status code returned by Cloudflare https://http.dev/524
I’m using Airbyte OSS. I suspect it might be due to the large number of table I have in BigQuery.
I experienced similar issues with BigQuery because schema discovery took ~17 minutes.
If you have Cloudflare in front of your Airbyte instance you need to check whether you can change timeouts
https://community.cloudflare.com/t/a-timeout-occurred-error-code-524-visit-cloudflare-com-for-more-information/541213
https://community.cloudflare.com/t/community-tip-fixing-error-524-a-timeout-occurred/42342
https://community.cloudflare.com/t/error-524-best-practice/469479/2
or perform changes when connection is not going through Cloudflare
maybe you can access your Airbyte instance via VPN or with port-forwarding?
Ah interesting. Indeed my Airbyte is set up behind Cloudflare. I’ll look into this. Thank you very much for the resources!