Setting AWS_ASSUME_ROLE_EXTERNAL_ID for S3 destination in self-hosted Airbyte

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

When trying to add an S3 destination with role-based authentication in self-hosted Airbyte, the user is unable to find a way to set the AWS_ASSUME_ROLE_EXTERNAL_ID environment variable as instructed in the documentation.


Question

I have a weird one. I’m on self-hosted airbyte trying to add an S3 destination. I want to use the roll based authentication method, but the docs are asking for me to set an environment variable in the worker:
https://docs.airbyte.com/integrations/destinations/s3#authentication-option-1-using-an-iam-role-most-secure
the var it is asking me to set is AWS_ASSUME_ROLE_EXTERNAL_ID="{your-external-id}" which if I run the destination manually with a docker run command and include that as a variable the check passes. But there is no way in the user interface to add a custom environment variable to a destination that I can see. The docs make it sound like it is super obvious… so I am confused. I am on the latest version (installed today)


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["s3-destination", "role-based-authentication", "environment-variable", "self-hosted", "latest-version"]
2

I’m not sure about that method, because of note in docs. It looks like Airbyte Cloud setup, not Airbyte OSS.

> NOTE
> S3 authentication using an IAM role member must be enabled by a member of the Airbyte team. If you’d like to use this feature, please <https://airbyte.com/company/talk-to-sales|contact the Sales team> for more information.

3

I use this approach

  annotations:
    <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: ${role_arn}```
as mentioned in this thread
<https://airbytehq.slack.com/archives/C021JANJ6TY/p1725899715948819?thread_ts=1725896626.586719&amp;cid=C021JANJ6TY>

This is for helm charts deployment, but your setup may be different.