Adjusting resources for sync jobs in Airbyte helm chart

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

The user is trying to adjust the resources assigned to sync jobs in Airbyte using a values.yaml file in the latest Airbyte version with the latest Airbyte helm chart. They are facing issues configuring containers like socat in destination and source pods.


Question

Hi everyone! I am working with latest airbyte version with latest airbyte helm chart.

I am trying to adjust the resources assigned to my sync jobs. This is my values.yaml file

  jobs:
    resources:
      requests:
        cpu: "3000m"
        memory: "7000Mi"
      limits:
        cpu: "3000m"
        memory: "7000Mi"
  env_vars:
    SOCAT_KUBE_CPU_REQUEST: "3"
    SOCAT_KUBE_CPU_LIMIT: "3"
    SIDECAR_KUBE_CPU_REQUEST: "3"
    SIDECAR_KUBE_CPU_LIMIT: "3"```
This result in my sync jobs main container configured with the resources I am setting up ( 3 cores and 7Gb Memory) but I am not able to configure the other containers like socat (relay-stdout and relay-stdin containers). This is only happening to me on `destination`  and `source` pods. `Check` pods are well configured.

Has anyone come across this recently?

With that `values.yaml` file I get this logs...

```Attempting to start pod = source-postgres-read-75-0-dxvvb for airbyte/source-postgres:3.4.21 with resources 
ConnectorResourceRequirements[
  main=io.airbyte.config.ResourceRequirements@11b2dabe[
    cpuRequest=3000m,
    cpuLimit=3000m,
    memoryRequest=7000Mi,
    memoryLimit=7000Mi,
    additionalProperties={}], 
  heartbeat=io.airbyte.config.ResourceRequirements@1038913f[
    cpuRequest=0.05,
    cpuLimit=0.2,
    memoryRequest=25Mi,
    memoryLimit=50Mi,
    additionalProperties={}], 
  stdErr=io.airbyte.config.ResourceRequirements@3035ec53[
    cpuRequest=0.01,
    cpuLimit=0.5,
    memoryRequest=25Mi,
    memoryLimit=50Mi,
    additionalProperties={}], 
  stdIn=null, 
  stdOut=io.airbyte.config.ResourceRequirements@176f79c1[
    cpuRequest=0.5,
    cpuLimit=1,
    memoryRequest=25Mi,
    memoryLimit=50Mi,
    additionalProperties={}]],
  
Attempting to start pod = destination-snowflake-write-75-0-zqfwh for airbyte/destination-snowflake:3.11.0 with resources 
ConnectorResourceRequirements[
  main=io.airbyte.config.ResourceRequirements@2b5ea88b[
    cpuRequest=3000m,
    cpuLimit=3000m,
    memoryRequest=7000Mi,
    memoryLimit=7000Mi,additionalProperties={}], 
  heartbeat=io.airbyte.config.ResourceRequirements@1038913f[
    cpuRequest=0.05,
    cpuLimit=0.2,
    memoryRequest=25Mi,
    memoryLimit=50Mi,
    additionalProperties={}], 
  stdErr=io.airbyte.config.ResourceRequirements@460bd062[
    cpuRequest=0.01,
    cpuLimit=0.5,
    memoryRequest=25Mi,
    memoryLimit=50Mi,
    additionalProperties={}], 
  stdIn=io.airbyte.config.ResourceRequirements@63845f91[
    cpuRequest=0.5,
    cpuLimit=2,
    memoryRequest=25Mi,
    memoryLimit=50Mi,additionalProperties={}], 
  stdOut=io.airbyte.config.ResourceRequirements@3d40a99a[
    cpuRequest=0.01,
    cpuLimit=0.5,
    memoryRequest=25Mi,
    memoryLimit=50Mi,
    additionalProperties={}]] ```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1720693225788099) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["adjusting-resources", "sync-jobs", "airbyte-helm-chart", "values.yaml", "containers", "source-pods", "destination-pods"]
</sub>