Summary
The user is facing an issue in the Docker Compose Airbyte container where it fails to create the intermediate container while running a data pipeline ‘Connection’ on Airbyte UI. The error is related to a cgroup error, possibly due to CPU and memory limits specified in the Docker run command.
Question
hello, i am getting an issue in my docker compose airbyte container where it is failing to create the intermediate container while running a data pipeline ‘Connection’ on Airbyte UI. On the data job log it says it is a cgroup error and digging further into the logs on the docker container i have found that it is failing because when it does this docker run command
--network host -v airbyte_workspace:/data -v oss_local_root:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-google-sheets:0.5.1 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e FIELD_SELECTION_WORKSPACES= -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE=dev -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=<http://host.docker.internal:4317> -e FEATURE_FLAG_CLIENT=config -e AIRBYTE_VERSION=0.61.0 -e WORKER_JOB_ID=1 --cpus=1 --memory-reservation=1Gi --memory=2Gi airbyte/source-google-sheets:0.5.1 read --config source_config.json --catalog source_catalog.json```
it is try to also specifying some cpu limits :
`--cpus=1 --memory-reservation=1Gi --memory=2Gi`
this is where i imagine the cgroup issue is coming up. has anyone seen this before? I would appreciate any help!
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1717195803882529) if you want
to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
["docker-compose", "airbyte-container", "cgroup-error", "data-pipeline", "cpu-limits", "memory-limits"]
</sub>