Error setting up Airbyte with AWS Secrets Manager

Summary

When trying to set up Airbyte with AWS Secrets Manager, encountering an ‘Internal Server Error’ related to the AWS region not being recognized correctly. Attempted to pass AWS_REGION through docker-compose without success. Seeking pointers on how the environment gets populated to the config.


Question

Hi folks! I’m trying to set up airbyte (running in docker inside ec2) with aws secrets manager, but having some issues. I was trying to comb through <https://docs.airbyte.com/operator-guides/configuring-airbyte|config instructions> and <https://www.restack.io/docs/airbyte-knowledge-airbyte-aws-secret-manager-integration|secrets manager setup>, but i’m still getting an error: Internal Server Error: Request endpoint must have a valid hostname, but it did not: <https://secretsmanager>..<http://amazonaws.com|amazonaws.com> when creating e.g. a destination that would store the secrets in secrets manager. Based on the url it looks like it’s not getting the aws region correctly. I’ve tried to pass in AWS_REGION and some permutations of it through docker-compose, but no luck. I’ve also tried to comb through some source code, but couldn’t pinpoint exactly how the environment gets populated to the config. Anyone have any pointers?



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["airbyte", "aws-secrets-manager", "internal-server-error", "docker-compose", "aws-region"]

And just to clarify I’m passing the region to worker and server, this error seems to come form airbyte-server

and I’m using the latest trunk (0.57.2) of airbyte

ok, looking at the Dockerfile, seems the variable is remapped: https://github.com/airbytehq/airbyte-platform/blob/main/airbyte-server/src/main/resources/application.yml#L124C19-L124C44

This seemed to resolve the issue