Best Place to Call Methods in Connector Development for REST API


In connector development for a REST API using Python Airbyte CDK, the user is seeking advice on the best place to call create, poll, and download methods to interact with the API for obtaining reports.


Hello :slightly_smiling_face: , I have hit a road block in my connector development for a REST API, looking for help. I have used python airbyte cdk for dev until now. I need to call create, poll and download methods to interact with the API to get the reports I want. Which is the best place inside the connector to call these methods?

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["connector-development", "rest-api", "python-airbyte-cdk", "create-method", "poll-method", "download-method", "reports"]

New message text here

<@U04SE163WC8> If the logic to insert, poll and download are in read_records() then when running the read method for a stream , the insertion is being called twice, how to avoid it

Hmm, I don’t know enough to give you a confident answer yet. However, you might find it helpful to review the Airbyte Documentation or post your question on the Airbyte Forum for more specific guidance.

<@U04SE163WC8> why is path and parse_response checked twice

A message from|

Based on the information provided in the extracts, it’s not clear why path and parse_response would be checked twice in your specific context. However, in general, methods like path and parse_response could be called multiple times in different parts of the code for various reasons such as error handling, data validation, or different stages of data processing. If you’re seeing these methods being called twice in a way that seems unnecessary or inefficient, it could be due to a bug or an oversight in the code. Without more specific information about the code and context, it’s hard to provide a more precise answer. For more detailed help, consider posting your question on the Airbyte Forum or checking the Airbyte Documentation.