Airbyte integration with Airflow by using IAP credentials

Q&A
#1

Hi, I added Google IAP for Airbyte authentication. When I try to connect from airflow, I get Invalid IAP credentials: empty token. Did anyone successfully connect the airbyte from airflow using IAP? Any inputs are appreciated

Thanks
Naresh

#2

Hi @nareshmeda,
Did you follow this guide on how to programmatically access IAP protected app?
I am not aware of your current network setup, but if Airflow and Airbyte share the same private network you might not need to reach Airbyte from Airflow on the internet and use your local network for intercommunication between these two apps.

1 Like
#3

Hi @alafanechere, Thank you so much for the response. Yes, both are in the same network. I tried with internal IP, and now we can trigger the connection from airflow. If we want to use programmatic access for an IAP-protected app, do we need to edit the airbyte operator code and add programatic authentication?

Naresh

#4

Hey @naresh, I’m glad to hear that you succeeded in connecting Airflow to Airbyte.

If we want to use programmatic access for an IAP-protected app,

I think following this guide will be sufficient and no operation on Airbyte itself should be required. You will have to send Authorization headers with the request you send to Airbyte’s API. But it will not work out of the box with Airflow as the Airbyte operator does not support custom HTTP headers.
My suggestions would be on of the following:

  • Contribute to the AirbyteHook to add custom headers
  • Use the HTTP operator and call the Airbyte API with this HTTP connector and pass the custom HTTP headers required by IAP.