- Is this your first time deploying Airbyte?: No
- Deployment: AWS ECS on EC2
- Airbyte Version: 0.50.1
Route53 DNS resolution
Hi Airbyte people!
I’m working on setting up Airbyte on AWS ECS with EC2 instances (Amazon Linux) and it actually works. Deployed with CloudFormation template generated with ‘docker compose convert’. Airbyte is up and running successfully untill you are trying to configure some source. The problem is that ‘Server’ can not resolve custom domain names and falls into following error after attempt to setup source.
WARNING: [Channel<11>: (airbyte-temporal.airbyte-platform.local:7233)] Failed to resolve name. status=Status{code=UNAVAILABLE, description=Unable to resolve host airbyte-temporal.airbyte-platform.local, cause=java.lang.RuntimeException: java.net.UnknownHostException: airbyte-temporal.airbyte-platform.local: Name or service not known
While same name can be successfully resolved from within EC2 instance or server container
bash-4.2# curl -v airbyte-temporal.airbyte-platform.local:7233
* Trying 10.0.1.13:7233...
* Connected to airbyte-temporal.airbyte-platform.local (10.0.1.13) port 7233 (#0)
> GET / HTTP/1.1
> Host: airbyte-temporal.airbyte-platform.local:7233
> User-Agent: curl/7.88.1
> Accept: */*
>
* Received HTTP/0.9 when not allowed
* Closing connection 0
curl: (1) Received HTTP/0.9 when not allowed
Work around can be done thru Airbye deployment with default names and mapping them in /etc/hosts
echo 'airbyte-temporal airbyte-temporal.airbyte-platform.local' >> /etc/hosts
This will solve the issue but it will not work for ‘webapp’ since by default it uses localhost. So it looks like Arbyte application doesn’t resolve DNS names thru /etc/hosts and simply uses default names from docker compose.
So the question is do we have any option to configure Airbyte with DNS resolution? It looks for now that this is the only blocker to deploy Airbyte to AWS ECS on top of EC2.