Is it possible to NOT use SharedMountVolumes in a Kubernetes Deployment?

marcosmarxm · March 25, 2022, 9:01pm

Not today, Airbyte use the Shared Volume to transfer some config files.

airbytehq/airbyte/blob/6aa77bdd60a842bf718ea062b7a2fd7bda96537f/airbyte-workers/src/main/java/io/airbyte/workers/process/KubePodProcess.java#L64-L95

      
        
            /**
             * A Process abstraction backed by a Kube Pod running in a Kubernetes cluster 'somewhere'. The
             * parent process starting a Kube Pod Process needs to exist within the Kube networking space. This
             * is so the parent process can forward data into the child's stdin and read the child's stdout and
             * stderr streams and copy configuration files over.
             *
             * This is made possible by:
             * <ul>
             * <li>1) An init container that creates 3 named pipes corresponding to stdin, stdout and std err on
             * a shared volume.</li>
             * <li>2) Config files (e.g. config.json, catalog.json etc) are copied from the parent process into
             * a shared volume.</li>
             * <li>3) Redirecting the stdin named pipe to the original image's entrypoint and it's output into
             * the respective named pipes for stdout and stderr.</li>
             * <li>4) Each named pipe has a corresponding side car. Each side car forwards its stream
             * accordingly using socat. e.g. stderr/stdout is forwarded to parent process while input from the
             * parent process is forwarded into stdin.</li>
             * <li>5) The parent process listens on the stdout and stederr sockets for an incoming TCP
             * connection. It also initiates a TCP connection to the child process aka the Kube pod on the
             * specified stdin socket.</li>

This file has been truncated. show original

Topic		Replies	Views
Adding Volume Mounts to Airbyte Worker Pods in Kubernetes Platform Questions kubernetes , platform , question , airbyte-worker , custom-source	2	112	May 16, 2024
K8S Production reliable deployement Q&A	2	632	March 30, 2022
Browsing-output-logs on Kubernetes Platform, Deploy & Infra Issues normalization , kubernetes , deploy	2	737	March 24, 2023
Airbyte port forwarding in Kubernetes namespace Platform, Deploy & Infra Issues kubernetes , deploy	2	1284	March 11, 2022
Best architecture for deploying Airbyte on local Kubernetes cluster for handling 500GB data volume Platform Questions kubernetes , deploy , platform , airbyte , data-volume	0	30	October 2, 2024

Is it possible to NOT use SharedMountVolumes in a Kubernetes Deployment?

Related topics