Summary
Airbyte custom API endpoint to fetch data from a source and return it in an HTTP response
Question
Is it possible to make airbyte expose an custom API endpoint which returns data fetched from a source?
My usecase is that I have an ecommerce which wants to check during registration process whether an user exists in 3rd party system.
The flow i imagine could be like this: ecommerce makes a http request to airbyte, airbyte fetches data from source, and returns the data in the http response.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["custom-api-endpoint", "data-fetching", "http-request", "source-data"]
No, Airbyte is really about batch workloads and that you’re describing is a transactional workload. It simply takes too long to spin up the connector just to check for an account.
I would recommend thinking about something like a Google Cloud Function or AWS Lambda that’s serverless that just makes the API call and passes back the result from the production system without fully exposing it. But you will want to put some safeguards in place to avoid undue load on the production system (e.g. if a bot is trying to create a thousand accounts per second)
Thanks for confirmation <@U035912NS77>!