Switching from minIO to s3 for job logging on Kubernetes with Helm deploys

Summary

Switching job logging from minIO to s3 on Kubernetes with Helm deploys, encountering ‘wrong region’ error on worker pod


Question

Does anyone have most updated/proven way to switch from minIO to s3 for the job logging? I am on kuberenetes and helm deploys. I keep getting different errors - the latest is “wrong region - should be eu-west-1”… All the errors keep occurring on the worker pod. Any up to date steps is appreciated :smile: - I am on latest helm release



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["switching-from-minio-to-s3", "job-logging", "kubernetes", "helm", "wrong-region-error", "worker-pod"]

check docs https://docs.airbyte.com/deploying-airbyte/on-kubernetes-via-helm#external-logs-with-s3

Hi <@U07AD3NSQS2> I provided the entire deployment codes for EKS, Airbyte, Postgres 16.3, all use Terraform at
https://github.com/hongbo-miao/hongbomiao.com/issues/17463#issuecomment-2180242019
And all issues I met and solutions in the first comment :relaxed:

It’s also worth noting that right now the <https://docs.airbyte.com/enterprise-setup/implementation-guide|Self-Managed Enterprise Kubernetes docs> are a little more accurate to the updated values.yaml format. Someone on the Airbyte team said they’re aware of this and are actively working on improving the deployment docs to line up with the most recent changes

(just make sure you don’t actually deploy the SME versions of the chart and don’t add the enterprise deployment type value)

Their team has been talking about a lot of changes which will bring all deployment types much closer in line, which will also help the docs be more consistent. So lots to look forward to in the coming weeks/months