Quick question, is it possible for Airbyte to work on K8S without a shared mount ?
From our Slack
Not today, Airbyte use the Shared Volume to transfer some config files.
/**
* A Process abstraction backed by a Kube Pod running in a Kubernetes cluster 'somewhere'. The
* parent process starting a Kube Pod Process needs to exist within the Kube networking space. This
* is so the parent process can forward data into the child's stdin and read the child's stdout and
* stderr streams and copy configuration files over.
*
* This is made possible by:
* <ul>
* <li>1) An init container that creates 3 named pipes corresponding to stdin, stdout and std err on
* a shared volume.</li>
* <li>2) Config files (e.g. config.json, catalog.json etc) are copied from the parent process into
* a shared volume.</li>
* <li>3) Redirecting the stdin named pipe to the original image's entrypoint and it's output into
* the respective named pipes for stdout and stderr.</li>
* <li>4) Each named pipe has a corresponding side car. Each side car forwards its stream
* accordingly using socat. e.g. stderr/stdout is forwarded to parent process while input from the
* parent process is forwarded into stdin.</li>
* <li>5) The parent process listens on the stdout and stederr sockets for an incoming TCP
* connection. It also initiates a TCP connection to the child process aka the Kube pod on the
* specified stdin socket.</li>
This file has been truncated. show original