Developing destination connector for SQL Server to REST API using Python CDK

Summary

Developing a destination connector from SQL Server to REST API using Python CDK


Question

in my scenario i have sql server as source and rest api as destination. So in this case i have to develop the destination connector with Python CDK ?



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["sql-server", "rest-api", "destination-connector", "python-cdk"]

Yes, you can develop a destination connector using the Python CDK (Connector Development Kit) provided by Airbyte. The Python CDK simplifies the process of writing connectors by providing base classes and helper functionalities for various features such as authentication, pagination, handling rate limiting, and caching. To start developing your connector, you can use the CDK template generator which creates all the required scaffolding, allowing you to focus on implementing the API. You can find a complete tutorial for implementing an HTTP source connector in this tutorial. Once you’ve implemented your connector, you can test it. There are three options for testing, one of which is the native python way which does not require any Docker or involvement of Airbyte itself yet. After testing, you can containerize your connector so you can use it in the Airbyte UI. You can either use airbyte-ci or build the docker image using your own Dockerfile. Please note that while you can write a connector from scratch in any language as long as it follows the Airbyte Specification, most contributors felt comfortable writing in Python, so a Python CDK was created to accelerate this development. Sources: - Python CDK - Connector Development - Testing Connector - HTTP-API-based Connectors - Reading Data from the Source

Yes, as of today — the Connector Builder cannot build destinations. In custom destinations, you’re on your own now ;( An area we’ll focus on later, though.