Setting up self-hosted Airbyte S3 source with IAM Role

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

User is facing issues setting up self-hosted Airbyte’s S3 source with IAM Role. Confirmed user can assume the role and read data from S3 bucket using AWS CLI, but encountering configuration check failure in Airbyte connector setup.


Question

Heya, I’m trying to set up self-hosted AirByte’s S3 source to work with the <S3 | Airbyte Documentation“IAM Role” setup> (the help looks slightly different in the AirByte UI than in the docs site).

I have the policy + role + trust relationship (Edit: ~including external ID~) + user and all that set up. I can use the AWS CLI to confirm that the user can assume the role and read data from the given S3 bucket.

Then trying to set things up in the AirByte connector it fails with a configuration check. Did anyone else made it work with the role assumption?

If I directly attach the role to the user, it all works, of course. I just wanted to check whether this is expected to work?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["self-hosted", "airbyte-s3-source", "iam-role", "configuration-check", "aws-cli"]
2

Actually, I see that external ID would need to be injected from the environment indeed, not sure why I thought it will be added by default.

But taking that bit out (and not setting a condition) I still encounter the error…