Moving Airbyte from Docker Compose to EKS K8s

Summary

User is transitioning Airbyte from a Docker Compose setup to an EKS Kubernetes environment and is facing issues with the configuration files not being saved in the expected workspace directory. They seek guidance on configuring the Helm chart to store JSON config files in Minio or another location.


Question

Hi all,
I’m moving my docker-compose Airbyte to EKS K8s
My docker-compose worker has:
WORKSPACE_ROOT=/tmp/workspace
And in my worker docker /tmp/workspace folder i have all the logs and json config files = GOOD

My K8s Helm chart worker pod has:
WORKSPACE_ROOT=/workspace
The worker pod has no files in /workspace :frowning:
The logs are in Minio object storage = good
But where are the json config files? :disappointed:
Please advise how to configure Airbyte Helm to save the json files to Minio object store or some other location.
Thanks.



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['docker-compose', 'eks', 'k8s', 'helm', 'workspace', 'minio', 'json-config']