- Is this your first time deploying Airbyte?: No
- OS Version / Instance: Linux
- Memory / Disk: 512GB
- Deployment: Kubernetes (Helm)
- Airbyte Version: Tried 0.44.2 & 0.44.4 (Helm 0.45.20 and 0.45.17)
I have a couple of issues.
1 - My storage layer is either Minio (local desktop) or Dell ECS (nonprod/prod). From what i understand, for both these we can use storageType=MINIO.
However, i am in a situation where both the endpoint and region needs to be configured for Minio and ECS.
Without region, the airbyte-server logs show a 400 BadRequest when talking to storage.
With region, the log4j2 appender doesnt get initialized due to validation failures which cite that only one of endpoint or region can be configured.
All other tools i use allow for using both parameters, i am not sure why there is this restriction/validation in Airbyte.
There is already a Github Issue (https://github.com/airbytehq/airbyte/issues/23261) for this but seems to be deprioritized (assuming “platform-hold” means its on hold). Can we expect a resolution soon?
2 - The state storage bucket is controlled by the following line in env-configmap.yaml:
STATE_STORAGE_MINIO_BUCKET_NAME: airbyte-state-storage
Can this bucket name be changed as per my requirement?
As there is no way to do this through values.yaml, i would end up customizing the helm chart files. What i want to understand is whether that customization could break something in code or not?
Also, is there an option of configuration at a folder level, instead of assigning a whole bucket, so i can use different folder withing the same bucket for state vs logs?