Exploring how to handle custom connectors using source control and Terraform for efficient management.
How can I manage custom built connectors with source control and terraform?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
<@U02TNBMDUMQ> and <@U071YUDEYHZ> as this could be better suited to have a how-to/tutorial in official documentation and not in the API doc.
Someone asked if we already have an endpoint to upsert a custom connector manifest into Builder and publish it. I don’t think we have, but I will ask around and see how much of a lift that would be.
We don’t provide a terraform plugin for this, and I’m not well versed in that area ;-(
Thank you! It would be super beneficial for our team
https://reference.airbyte.com/reference/using-the-terraform-provider#3-create-a-source there is one example using custom connector
You need to create the connector manually and later reference the uuid for that custom connector in your Terraform resource.
Good to know. So the workflow could be creating the custom connector downloading the yaml config and then uploading to the prod and dev airbyte instances. The only challenging part is it looks like the custom connector id will vary from instance to instance which makes CI/CD a little more challenging
Yes, I totally agree with you. I raised this to the engineering team responsible for the API that resources (sources and destinations) should have a readable identifier to make CI/CD and Terraform pipeline more robust and easy.