Summary
Configuring job securityContext in Kubernetes for Airbyte connectors
Question
hello community, does anyone know how i can configure the job securityContext in kubernetes? From the chart configuration, we can configure global.jobs.kube.labels for example but not the securityContext and this is a problem given i have gatekeeper preventing to deploy pods if the securityContext is not explicit. This prevents me to configuring a new source, a new connector, a new destinations, etc. The AI assistant <#C01AHCD885S|ask-ai> did not provide any answers except that it doesn’t seem that’s possible from the configuration. From what i have seen from airbyte source code, it doesn’t seem to be a yaml template but rather generated directly from the code
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["kubernetes", "job-securitycontext", "airbyte-connectors", "gatekeeper"]
Does this not inherit the containerSecurityContext/podSecurityContext set on worker?
Do you have a link to the source code you’re looking at?
no, because i alreayd have containerSecurityContext set on my values.yaml file, which is required to explicitely set the security context on the worker pod. But it doesn’t have any impact on the pods created by the worker itself. This seems to be managed by the global.jobs.* in the <Artifact Hub definition> like the labels, resoucres, etc that i also had to set in order to be compliant with the gatekeeper installed on the cluster. But there is no field to manage the securityContext directly.
I have found <airbyte-platform/airbyte-workers/src/main/resources/application.yml at d7feda1988931d691ea66acd303e0c3025edfa56 · airbytehq/airbyte-platform · GitHub part of the code> that doesn’t seem to allow us to modify the securitycontext of the pods generated by the worker
for those who might have a similar issue, i was able to make it work using airbyte >0.57.0 at least (i was using 0.56 before). This application version has a code change that <airbyte-platform/airbyte-commons-worker/src/main/java/io/airbyte/workers/process/KubePodProcess.java at d4e64e0a64c6f7ff72e19d6890be5c747f902f70 · airbytehq/airbyte-platform · GitHub the security context somewhat configurable>. It is not possible to specify the values, but by using the environment variable ROOTLESS_WORKLOAD (not documented as of today), a securityContext with rootless configuration will be injected into the pods created by the worker.
Great find; definitely worth putting in a feature request for a more explicit configuration on these since it can be really hard to deal with locked down environments otherwise. I feel like if they mirrored the way that the jobs config works for resource requests/limits it would make a lot of sense.