EC2 Storage Management for Airbyte

Summary

User seeks assistance in managing storage space on an EC2 instance running Airbyte, currently utilizing a values.yaml file for configuration. They are manually deleting old files to control storage and inquire about better management practices.


Question

I’m having issues with keeping the EC2 instance that is running Airbyte under control on storage space. I have a values.yaml file with the below content, but I am currently going about once every two weeks and just doing a mass delete of files older than 14 days. Is there something else that I need to do to help keep the EC2 storage space under control?

  env_vars:
    TEMPORAL_HISTORY_RETENTION_IN_DAYS: 7```
I installed Airbyte using the below command:
```abctl local install --values ./values.yaml ```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1733431628719549) if you want
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
['ec2', 'storage-management', 'values.yaml', 'abctl', 'temporal-history-retention']
</sub>

Are you using an external database <@U071NLJCZLL> or default airbyte-db service?

<@U01MMSDJGC9> I’m using the default Airbyte-db service