I have published a tutorial called Build a connector to extract data from the Webflow API, that goes into detail about the development of a custom Airbyte connector. Interesting topics that are discussed include:
- Manually interacting with HTTP APIs.
- The design of an Airbyte (Python) source connector.
- Dynamic stream creation and automatic schema extraction.
- How to build a new source connector from scratch – using the Webflow source connector implementation as an example.
The Webflow connector that is presented is used at Airbyte to extract data from Webflow (our CMS) and drive it into BigQuery to improve our analytics capabilities.
If you are interested in writing your own code to extract data from APIs, you will likely find the tutorial to be helpful.
Another interesting aspect of this tutorial is that it shows how to query an API for data, and then use the data from the first query in a subsequent API call.
Specifically the list of collections available in Webflow is retrieved in a first API call, and a subsequent API call uses the collection Ids (from the list returned by the first API call) to construct the URL that is used retrieve the contents of each collection. Additionally, the schema for each collection is retrieved in another API call, using Ids returned in the first API call.
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.