IAM role authentication for S3 source in Airbyte OSS

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

IAM role authentication for S3 source in Airbyte OSS is not working as expected. User is facing issues with the IAM role setup in the Source check method. There is an open issue on GitHub related to this problem.


Question

Good day. Is anyone successfully using IAM role authentication for an S3 source in Airbyte OSS? I have configured the IAM roles and policies such that I can assume the relevant role and list the contents of the s3 bucket I want to read from from the command line of the EC2 instance Airbyte is deployed on but the same role is failing the check method in Source setup. I note this <Issues · airbytehq/airbyte · GitHub issue> from <@U062JJE1QCX>, which was most recently tagged Icebox but remains open. I will move on from this approach if it does not work in OSS but would prefer it to keypair authentication if that is an option.


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["iam-role-authentication", "s3-source", "airbyte-oss", "open-issue", "keypair-authentication"]
2

Just want to mention - this was never resolved for us either. We ended up moving our ingestion jobs from S3 to use Google Storage Transfer service instead.