Clarification on whether connectors built using the connector builder can be used for both Source and Destination
Hi All,
I’ve a small question:
Can connectors been build by the connector builder only be used for Source and not as Destination?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Currently it is only for source. Destination there are a lot of complexity to ensure they are able to process the data (data mapping-typing, deduping, truncating, etc). There are a lot of advances in the Java/Kotlin Destination CDK. For now it isn’t in the roadmap to provide the configuration of new destination using the builder.
Thanks!
I’ve seen a message about “Over the next few months, the project will only accept connector contributions that are made using the <https://docs.airbyte.com/connector-development/config-based/low-code-cdk-overview|Low-Code CDK> or the <https://docs.airbyte.com/connector-development/connector-builder-ui/overview|Connector Builder>…”
Does this mean that we can’t create our own Destination connector unless we use the Low-Code CDK for Airbyte?
Just to give you some small background: I’m working for Marigold (Selligent). And i’m currently investigating on how we can build/create a connector Source & Destination.
We have a bunch of community Python destinations, but providing maintenance and support for them is a challenge. You can build a custom destination and provide the docker image to your customers / Airbyte users who can deploy it as a custom destination connector. A few partners such as Planetscale do that.
We’re figuring out the story of how to support Python destinations properly.
Hi Natik,
Thank you for your response, but does it mean that ‘for now’, it’s not possible to get it within the certificate/community list. Only possible for clients that are not using the cloud one?
As of today, you’re right. But, it’s a problem on my radar, I’m looking for a way to figure it out.
The problem is once it’s merged, it’s difficult to maintain. But there are companies like yours who in theory want to maintain and resolve bugs in their destinations.
I would love to work with you on this if you’re interested.
Hi Natik,
Thank you for your response. I’ll discuss it with with my manager