Error creating connection in Airbyte on Kubernetes

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

When trying to create a connection in Airbyte on Kubernetes, a KubernetesClientException is thrown due to missing CPU and memory limits and requests for connector-sidecar, init, and main containers.


Question

hello community,
I am new to airbyte and trying to deploy airbyte community edition on kubernetes.
I am able to deploy and all pods are up, but when I am try to create connection i am getting below error in workload-launcher.
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PATCH at: <https://10.100.0.1:443/api/v1/namespaces/data-engineering/pods/e-mongodb-v2-check-33a37202-430d-4270-b979-6c008749a86d-0-lnidc?fieldManager=fabric8>. Message: pods "e-mongodb-v2-check-33a37202-430d-4270-b979-6c008749a86d-0-lnidc" is forbidden: failed quota: data-engineering: must specify limits.cpu for: connector-sidecar,init,main; limits.memory for: connector-sidecar,init,main; requests.cpu for: connector-sidecar,init,main; requests.memory for: connector-sidecar,init,main. Received status: Status(apiVersion=v1, code=403, details=StatusDetails(causes=[], group=null, kind=pods, name=e-mongodb-v2-check-33a37202-430d-4270-b979-6c008749a86d-0-lnidc, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=pods "e-mongodb-v2-check-33a37202-430d-4270-b979-6c008749a86d-0-lnidc" is forbidden: failed quota: data-engineering: must specify limits.cpu for: connector-sidecar,init,main; limits.memory for: connector-sidecar,init,main; requests.cpu for: connector-sidecar,init,main; requests.memory for: connector-sidecar,init,main, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Forbidden, status=Failure, additionalProperties={}).
at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:660) ~[kubernetes-client-6.12.1.jar:?]
at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.requestFailure(OperationSupport.java:640) ~[kubernetes-client-6.12.1.jar:?]
at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.assertResponseCode(OperationSupport.java:589) ~[kubernetes-client-6.12.1.jar:?]
at io.fabric8.kubernetes.client.dsl.internal.OperationSupport.lambda$handleResponse$0(OperationSupport.java:549) ~[kubernetes-client-6.12.1.jar:?]

i am using helm chart version 1.0.0

i can not find any properties which can specify limits for connector-sidecar. i tried suppling below extraEnv in workload-launcher but it didnt helped.

  JOB_MAIN_CONTAINER_CPU_LIMIT: '200m'
  JOB_MAIN_CONTAINER_CPU_REQUEST: '250m'
  JOB_MAIN_CONTAINER_MEMORY_LIMIT: '1Gi'
  JOB_MAIN_CONTAINER_MEMORY_REQUEST: '256Mi'```
does anybody have faced somthing like this?

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1731324661785739) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["airbyte", "kubernetes", "connection", "error", "kubernetesclientexception", "cpu-limits", "memory-limits", "connector-sidecar"]
</sub>
2

Thanks for the reply <@U07S3DM6CTY>
I have already added this properties in http://global.jobs|global.jobs . but the error remains same.

3

Describe the replica pod which is created while sync happens
check that the envs which you updated is really updated in describe section
If env is updated then it might be resource issue so your pod is pending
if env is not updated recheck your values.yaml config

4

env are updated Rajath.

5

Then I am not sure
when I updated envs I was able to do it with helm installation in kubernates
I also tried abctl command to install
(here inside docker container cluster is running)
with abctl also I am getting same issue as yours
But helm installation I was able to do

6

For helm installation I am using airbyte version 0.64.5
for abctl installation it is automatically taking latest version 1.1.0
Either there might be extra configuration in latest version or there is a bug in latest version
I was able to make it work with 0.64.5
So let me know if you find anything useful
Thank you

7

Hi Shubham Kanungo,
I also faced same issue
you will have to update the job section in values.yaml something like below

    resources:
      requests:
         memory: 500Mi
         cpu: 300m
    
      limits:
         cpu: 300m
         memory: 750Mi```
The above envs are present in arbyte-env configmap

you can check env-configmap.yaml  file present present inside templates repo of airbyte-platform to get morew info on how envs are mapped to values.yaml
Thank you hope this helps