Issue with S3 Destination Role ARN not being used

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

The user is facing an issue where the Role ARN specified in the S3 destination configuration is not being used, causing the AssumeRole operation to be done to the wrong AWS account. This results in an Access Denied error when trying to connect to the S3 bucket.


Question

Hello team, I have a problem using the S3 destination, when i use the Role ARN field, the AWS account number is not being used.
My infra is:
• Account A
◦ my account
◦ existing role
◦ bucket i want to connect
• Account B
◦ role i want to use (same name as Account A role)
When my SRE team looked at the logs looks like the AsumeRole operation is done to Account A which does not have access to the buckets instead of the role in Account B that is the one I specify in the Role ARN field. The accounts are linked properly and in other services I can perform this operation perfectly, but in Airbyte I am getting a:

Could not connect to the S3 bucket with the provided configuration. Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: XXXXXX; S3 Extended Request ID: XXXX-extended; Proxy: null)```
Have anyone experienced a similar issue? In case this is a bug where can i report it? Thanks!

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1721036962162619) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["s3-destination", "role-arn", "assume-role", "access-denied", "bug"]
</sub>
2

I opened a <Issues · airbytehq/airbyte · GitHub issue>, in case anyone have the same issue. Thanks!

3

What setup do you have?
Helm charts + EKS?
docker compose + EC2?

I have Helm charts + EKS and service account uses role by calling sts:AssumeRoleWithWebIdentity

4

I am testing everything in local… so its the abctl local install trying to connect to AWS

5

do you use IAM user’s credentials locally?
because logic here https://github.com/airbytehq/airbyte/blob/91b290d372425b27dd6866258cb011e638f75a53/airbyte-cdk/java/airbyte-cdk/s3-destinations/src/main/kotlin/io/airbyte/cdk/integrations/destination/s3/S3DestinationConfig.kt#L354-L367|S3DestinationConfig.kt#L354-L367 determines how credentials are selected

I’d suggest to test things directly on AWS, because it’s harder to simulate similar behaviour locally, maybe localstack can help a bit with that

6

I usually connect using my credentials and setting the role i need… for what I see in that code if my credentials are set (they are) the role will never be taken into account. Am I right?

7

Yes. That’s why I suggested testing on AWS directly.