Deploy Kubernetes: JOB_KUBE_TOLERATIONS with no effect

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Linux
  • Memory / Disk: -
  • Deployment: Kubernetes
  • Airbyte Version: What version are you using now? 0.39.32-alpha
  • Source name/version: Postgres 0.4.30
  • Destination name/version: GCS 0.2.9
  • Step: Sync
  • Description:

I want to assign a toleration to the airbyte worker jobs with this value

        - key: k8s.company.com/node-pool
          value: data
          operator: Equal

However, if I don’t specify an effect this toleration is not added to the job.
like JOB_KUBE_TOLERATIONS: key=k8s.bodyguard.ai/node-pool,operator=Equal,value=data

And if I add an effect like
JOB_KUBE_TOLERATIONS: key=k8s.bodyguard.ai/node-pool,operator=Equal,effect=NoExecute,value=data

I got this error

1 node(s) had taint {k8s.bodyguard.ai/node-pool: data}, that the pod didn't tolerate

Is it possible to make this effect property optional ?

Thanks

Hey, you need to assign first the tolerations to the node so that while we are creating the pods it selects based on that. You can find it more here Taints and Tolerations | Kubernetes

It will also be helpful if you can help us with the usecase on how you are using TOLERATIONS.

Ok will have a look and will describe the use case if it’s successful but in a few word my airbyte pods are set to a particular node pool and I want that the sync worker job to be executed in this node pool and not in the default one

@harshith it works well with the good taint on the node, and the variables JOB_KUBE_NODE_SELECTORS and JOB_KUBE_TOLERATIONS properly set !