Airbyte integration with Airflow by using IAP credentials

Q&A
1

Hi, I added Google IAP for Airbyte authentication. When I try to connect from airflow, I get Invalid IAP credentials: empty token. Did anyone successfully connect the airbyte from airflow using IAP? Any inputs are appreciated

Thanks
Naresh

2

Hi @nareshmeda,
Did you follow this guide on how to programmatically access IAP protected app?
I am not aware of your current network setup, but if Airflow and Airbyte share the same private network you might not need to reach Airbyte from Airflow on the internet and use your local network for intercommunication between these two apps.

1 Like
3

Hi @alafanechere, Thank you so much for the response. Yes, both are in the same network. I tried with internal IP, and now we can trigger the connection from airflow. If we want to use programmatic access for an IAP-protected app, do we need to edit the airbyte operator code and add programatic authentication?

Naresh

4

Hey @naresh, I’m glad to hear that you succeeded in connecting Airflow to Airbyte.

If we want to use programmatic access for an IAP-protected app,

I think following this guide will be sufficient and no operation on Airbyte itself should be required. You will have to send Authorization headers with the request you send to Airbyte’s API. But it will not work out of the box with Airflow as the Airbyte operator does not support custom HTTP headers.
My suggestions would be on of the following:

  • Contribute to the AirbyteHook to add custom headers
  • Use the HTTP operator and call the Airbyte API with this HTTP connector and pass the custom HTTP headers required by IAP.
5

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.