Summary
The user is facing a ‘java heap space’ error when syncing data from DynamoDB using Airbyte deployed on an EC2 instance with a specific configuration. They are unsure how to increase the worker’s capacity.
Question
When sync from Dynamodb, i am having java heap space error. The airbyte is deployed on EC2 instance using docker with instance type of t2.2xlarge. Below are my configuration on .env files. From the docker stats, it seems the worker is overworking but don’t know how to increase it.
# Relevant to scaling.
SYNC_JOB_MAX_ATTEMPTS=3
SYNC_JOB_MAX_TIMEOUT_DAYS=3
SYNC_JOB_INIT_RETRY_TIMEOUT_MINUTES=5
JOB_MAIN_CONTAINER_CPU_REQUEST=0.7
JOB_MAIN_CONTAINER_CPU_LIMIT=0.8
JOB_MAIN_CONTAINER_MEMORY_REQUEST=25g
JOB_MAIN_CONTAINER_MEMORY_LIMIT=30g
NORMALIZATION_JOB_MAIN_CONTAINER_MEMORY_LIMIT=
NORMALIZATION_JOB_MAIN_CONTAINER_MEMORY_REQUEST=
NORMALIZATION_JOB_MAIN_CONTAINER_CPU_LIMIT=
NORMALIZATION_JOB_MAIN_CONTAINER_CPU_REQUEST=```
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1723191545835139) if you want
to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
["java-heap-space", "syncing", "dynamodb", "airbyte", "ec2-instance", "docker", "worker", "configuration"]
</sub>