Summary
Exploring the best method to fetch data from one API and use the result to fetch data from another API in Airbyte
Question
Hey Team, what is the best way using airbyte to fetch data from one api and use the result of first to fetch data from another api ?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here.
Join the conversation on Slack
Click here if you want to access the original thread.
`["fetching-data", "multiple-apis", "airbyte"]`
Airbyte connectors currently operate independently. For a more customized version, you’ll need to wrap the first API in a custom component and the data to the second API. There’s no guide for this, so you’ll need to study how connectors function to make it work :octavia-muscle: .
<@U05B037SDN3> when you say one api and use the result of first to fetch data from another api Do you really mean different API altogether or do you mean different calls or paths of an API where the first data is needed in order to sync the second data?
Either way you could ?maybe? create a hybrid connector that has both and use parent streams to get the data from the “first api”
2 different apis. Bit new with cdk of airbyte . I might need to do the thing you are saying
You can maybe pull it off with connector builder and not need to do it in CDK.
How to pull out the data from first api result and provide it to next api call (stream or new api ). How we can pass it as parameters
I have a case where I need multiple “values” from the parent stream so I hacked around and did this (I have no idea why I can’t reference values from parent stream with something like {{ stream_partition.parent_stream.blah }} as it seems trivial. But none the less this is how I did it (but not great)
Interesting. Let me try this if it works for my case