Airbyte CICD Best Practice

Hi Folks,

I’m curious as to the approaches to SLDC people are using, i’m specifically interested in using Azure DevOps pipelines. I can see that we can use Octavia CLI to maintain a repo of configs that could could be deployed from dev to prod airbyte instances, assuming state files would be ignored. Are people also using airbyte API to stop existing connections before applying updates? Bit confused.

Thanks,
Y

What is SLDC? You can check this repo https://github.com/airbytehq/airflow-summit-airbyte-2022 it’s using Airbyte/Airflow/dbt with CI/CD

Hi,
SDLC - Software Development Lifecyle

So what i’m thinking is, having two instances of Airbyte;
dev_test
prod

A manual SDLC would involve setting up dev sources, dev destinations and connections in UI or via Octavia CLI in dev_test, complete testing (for example check data in destination). Then in order to push to production we would mimic the above but use prod sources, prod destinations in our prod instance.

I’m curious as to how people are applying devOps for this process. I’m conscious that the export config > import config wouldn’t work here as we would need to change source/destination references from dev to prod and we would not want any state information overwritten in prod with dev source state information.

Perhaps i’m over complicating things in my head…

Yep, did you check the repo I share with you? It’s one example where we can have a dev/prod with same connectors and different credentials

Ah right. Thanks I’ll review this evening. :+1: