Error connecting Airbyte with Airflow using abctl

Slack Knowledge Base Q&A Connector Questions
, , , ,
41

I think airflow should be connected to bridge network, so host.docker.internal can work

42

maybe something like this?

      - bridge```
43

sorry for the delay… I tried, but i’m using other external networks to connect with minio and nessie

44

so you can add bridge to existing list

45

maybe something I did wrong? airflow deploy? some iptables rule?

46

I’d simplify airbyte ingress
It is possible to edit ingress like this
docker exec -it airbyte-abctl-control-plane kubectl edit ingress ingress-abctl -n airbyte-abctl

can you copy what content you get in editor?

47

vim should be default on abctl?

48

yup, it looks like vim is default editor

49

It was because of the host config

50

I reinstalled the abctl and set the default IP… it connected

51

Dude, thank you so much for all your help :heart:

52

Cool :sunglasses: Yup, this is because of this code https://github.com/airbytehq/abctl/blob/9b2585feeb5ed0e68f91064a48d431bdc96803d0/internal/cmd/local/local/specs.go#L16-L30|specs.go#L16-L30 in abctl repo
If you don’t provide --host option, then * is used, which “catches” all the hosts as long as you connect to nginx
If you provide --host option, then you have that value, plus localhost and host.docker.internal and you have to use those hosts when connecting to nginx; otherwise 404 is returned

53

Thank you so much for the explanation! One funny thing that I noticed is that, the connection didn’t work when I set the port, like on the airflow docs for airbyte: https://airflow.apache.org/docs/apache-airflow-providers-airbyte/stable/connections.html

It only worked setting the container name plus the endpoint

54

like this: http://airbyte-abctl-control-plane/api/public/v1/health

55

https://airflow.apache.org/docs/apache-airflow-providers-airbyte/stable/connections.html

If you are using Airbyte OSS: <http://localhost:8000/api/public/v1/>
Keep in mind that this documentation is for Airflow running on host machine, but not as Docker container connected to kind network

Well, this url was mentioned by me in conversation https://airbytehq.slack.com/archives/C021JANJ6TY/p1729613322246519?thread_ts=1729512052.391239&cid=C021JANJ6TY
It’s all about mapping different ports and what dns names you use and wheere
from host machine you access localhost:8000 (host: localhost, port: 8000) but when you “inside” Docker, running Docker container connected to kind network, you can use airbyte-abctl-control-plane (host; airbyte-abctl-control-plane, port 80)

56

thanks for all the content :heart: