Issue with Docker Compose Airbyte Container and cgroup error


The user is facing an issue in the Docker Compose Airbyte container where it fails to create the intermediate container while running a data pipeline ‘Connection’ on Airbyte UI. The error is related to a cgroup error, possibly due to CPU and memory limits specified in the Docker run command.


hello, i am getting an issue in my docker compose airbyte container where it is failing to create the intermediate container while running a data pipeline ‘Connection’ on Airbyte UI. On the data job log it says it is a cgroup error and digging further into the logs on the docker container i have found that it is failing because when it does this docker run command

--network host -v airbyte_workspace:/data -v oss_local_root:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-google-sheets:0.5.1 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e FIELD_SELECTION_WORKSPACES= -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE=dev -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=<http://host.docker.internal:4317> -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.61.0 -e WORKER_JOB_ID=1 --cpus=1 --memory-reservation=1Gi --memory=2Gi airbyte/source-google-sheets:0.5.1 read --config source_config.json --catalog source_catalog.json```
it is try to also specifying some cpu limits :

`--cpus=1 --memory-reservation=1Gi --memory=2Gi` 

this is where i imagine the cgroup issue is coming up. has anyone seen this before? I would appreciate any help!



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here]( if you want 
to access the original thread.

[Join the conversation on Slack](

["docker-compose", "airbyte-container", "cgroup-error", "data-pipeline", "cpu-limits", "memory-limits"]