Situation: we spun up a new Airbyte deployment (using Kubernetes, although depending on solution I’m thinking this could be applicable to other deployment types) and now need to migrate all of our current config (sources, destinations, connections, stream states, etc) to a new deployment.
The API could get us most of the way there (IE for each connection: wait til not running, stop it, transfer connection info to new deployment, delete old connection), but there’s two downsides that I can think of for using the API:
- You’d lose your state for incremental streams/have to re-backfill those (as I’m not seeing a way to update connection state via the API)
- You’d lose your job sync history/logs
Given that, I’m hoping most of this info is set in some config files somewhere and can essentially just be copied to the new deployment along with the logs. Given we’re using Kube, I’m assuming this isn’t quite as straightforward… Any thoughts on doing so?
