Summary
When trying to create a new connection in Airbyte using a Source (Netsuite sandbox) and destination (local JSON files) in a local Docker Compose deployment, it returns errors with http.badGateway. Need to determine if the tests of the source and destination are accurate.
Question
Just getting started and using the local Docker Compose deployment. I added a Source (Netsuiite sandbox) and destination (local JSON files) and these tested successfully,
When trying to use these for a new connection, it spins a bit and returns errors.http.badGateway
Are the tests of the source and destination accurate or should I revisit those?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["error", "http-badgateway", "new-connection", "source", "destination", "local-docker-compose", "netsuite-sandbox", "local-json-files"]
I would probably start by making sure that system-level (or load balancer) timeouts are set appropriately high. When a connection is set up, it has to spin up the workers and such, which takes a bit. then do those tasks and report back. A lot of platforms have default timeouts of as little as 30 seconds, so timeouts will almost always be an issue. I’d recommend at least 600 seconds to start.
Thank you Justin, I’ll have to dig into the Docker compose containers to see if ther’s a timeout being set.
Oh, I missed the word “local” in the original post. Shouldn’t be many issues with timeouts there unless you’re using the nginx config.
What version of Airbyte are you running? And is the error in the Webapp UI or in the connection logs?
And are you connecting on port 8000 or forwarding 80 through a proxy?
I believe the default nginx proxy used for auth when installing via Docker can be configured via the BASIC_AUTH_PROXY_TIMEOUT entry in .env. Usually that’s set to 900 seconds which is a pretty sane default. If you’re not setting it or commented out that line, it could be falling back to some other default value (e.g. 60 seconds). You can see how it’s used in the default configs https://github.com/airbytehq/airbyte-platform/blob/main/airbyte-proxy/nginx-auth.conf.template|here.
My Airbyte containers are all 0.58.1 and all at default as this is my first try using the system. I’m not ruling out a NetSuite problem but I did follow the instructions meticulously for the NetSuite Connector.