Octavia CLI connect to Airbyte instance through bastion host

Connector Questions & Issues
1

I have Airbyte deployed on an EC2 inside a private subnet that I connect to via a bastion / jump host.

To connect to the Airbyte instance from my local machine, I set up a local port forwarding session like this:

ssh -L 8000:<airbyte_ec2_instance>:8000 <bastion_ec2_user>@<bastion_host_ec2_instance> -i <file/path/to/bastion/pem/file> -N -f

The above has been working fine and I’m able to access the Airbyte UI by going to http://localhost:8000 in my browser.

I am now trying to install and use the octavia-cli.

When trying to run any of the octavia cli commands like ‘octavia list’ or ‘octavia init’, I get the following connection error:

Error: Could not reach your Airbyte instance, make sure the instance is up and running and network reachable: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /api/v1/health (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0xffff871d9e50>: Failed to establish a new connection: [Errno 111] Connection refused'))

I’ve confirmed that I’m able to reach http://localhost:8000/api/v1/health from my browser and also using Python’s requests library.

Is it currently not possible to use the octavia cli with my current Airbyte deployment (i.e. - jumping through a bastion host)? Or is there something I’m missing?

1 Like
2

Hi @pmehta , unfortunately Octavia CLI doesn’t support this out of the box. A possible solution would be to forward the traffic received by the bastion host on port 8000 to the Airbyte instance. Octavia is using the same port as the Airbyte instance, this isn’t a problem specific to Octavia, jumping through servers like this requires additional networking configuration.

Let me know if this helps!

3

Hi @natalyjazzviolin, thanks for the response.

Maybe I should clarify – before attempting to run any octavia cli commands, I’m already running the below ssh command to set up a local port forwarding session that forwards traffic from my localhost on port 8000 to the Airbyte instance on port 8000 using my bastion host as a jump server:

ssh -L 8000:<airbyte_ec2_instance>:8000 <bastion_ec2_user>@<bastion_host_ec2_instance> -i <file/path/to/bastion/pem/file> -N -f

I know this local port forwarding session is working correctly as I’m able to access the Airbyte UI from my browser by going to http://localhost:8000 and I can also successfully hit the api/v1/health endpoint using Python’s requests library and as well as via a curl command.

The problem is that it’s only the octavia cli that is unable to make the connection to the Airbyte instance and api/v1/health endpoint and I don’t understand why as like I said I’m able to reach the Airbyte instance and api/v1/health endpoint just fine via other means (from a web browser, python requests, curl command, etc.) after starting the local port forwarding session.

Is there any additional configuration like http headers or something similar that needs to be set for the octavia cli to make the connection?

4

I was finally able to get it working.

I had to set the environment variable “AIRBYTE_URL” to http://host.docker.internal:8000 in the octavia environment file.

Not sure why this was the solution but hey it’s working :man_shrugging:t5:

Edit: I think using “host.docker.internal” may only be applicable to people who have Docker Desktop running on their machines.

1 Like
5

Glad to hear it’s working for you now!

6

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.