I have Airbyte deployed based on the Docker guidance. After a while the workspace started to use up too much space that was causing issues. In another other post I’ve come across the MINIMUM_WORKSPACE_RETENTION_DAYS, MAXIMUM_WORKSPACE_RETENTION_DAYS, and MAXIMUM_WORKSPACE_SIZE_MB env vars that seem to configure the retention and seems like the thing I’m looking for. It’s not clear, however how to get those configurations applied so any cleanup happens?
From looking around GitHub issues and so on, I’ve seen that these variables might be used by the airbyte-scheduler, but that’s not used in the default Docker Compose-based deployment. Just applying these values to the airbyte-server and airbyte-worker doesn’t seem to make any difference.
Have any guidance on this?
Is this your first time deploying Airbyte?: Yes
OS Version / Instance: Debian
Memory / Disk: you can use something like 4Gb / 8GB