Automating Connector Creation with YAML File Documentation

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

Guide on automating the creation of a connector using a YAML file for Airbyte deployment in a cluster with Flux. Includes custom connector YAML files, HelmRelease, and HelmRepository configurations.


Question

Hi folks :slightly_smiling_face: could someone point me in direction of documentation on how to automate the creation of a connector using a YAML file?

I am deploying airbyte in a cluster using Flux. I have two custom connector yamls I wish to use. I have the following files/structures, and generic HelmRelease and HelmRepository files.

├── connector_source1.yaml
├── helm_release.yaml
└── helm_repository.yaml```

```helm_repository.yaml

apiVersion: <http://source.toolkit.fluxcd.io/v1beta2|source.toolkit.fluxcd.io/v1beta2>
kind: HelmRepository
metadata:
  name: airbyte-repo
  namespace: flux-system
spec:
  interval: 1h
  url: <https://airbytehq.github.io/helm-charts>```
```helm_release.yaml

apiVersion: <http://helm.toolkit.fluxcd.io/v2beta1|helm.toolkit.fluxcd.io/v2beta1>
kind: HelmRelease
metadata:
  name: airbyte
  namespace: airbyte
spec:
  interval: 5m
  install:
    timeout: 10m
  chart:
    spec:
      chart: airbyte
      version: 0.94.1
      sourceRef:
        kind: HelmRepository
        name: airbyte-repo
        namespace: flux-system
      interval: 5m
  values:
    airbyte:
      worker:
        resources:
          requests:
            memory: 1Gi
            cpu: '2'
          limits:
            memory: 2Gi
            cpu: '2'
      webapp:
        resources:
          requests:
            memory: 1Gi
            cpu: '2'
          limits:
            memory: 2Gi
            cpu: '2'```

Any help here would be appreciated, thanks!

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1717156414842469) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["automate-connector-creation", "yaml-file", "airbyte-deployment", "flux-cluster", "custom-connector", "helmrelease", "helmrepository"]
</sub>