Using Service Accounts for AWS Authentication with Airbyte

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Inquiring about the possibility of using service accounts for authentication with AWS in Airbyte.


Question

Does anyone know if its possible to use service accounts for authentication with AWS with airbyte?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["service-accounts", "authentication", "aws-connector", "airbyte-platform"]
2

for some functionality, but not for all yet. we use this for core airbyte functionality (secrets manager, logs) and the s3 source. redshift destination requires access key still

3

Yah thats what I was looking for. Is there any documentation or a slack post showing how to configure this? The helm chart is kinda lacking in terms documentation.

4

i tend to just use the helm chart source as documentation :joy:

5

Ok, I’ll go make some more coffee and try to suss it out.

6

here’s enough to get you started:

  image:
    tag: ${airbyte_version}
  storage:
    type: s3
    bucket:
      log: ${s3_logs_bucket}
      state: ${state_bucket}
    s3:
      authenticationType: instanceProfile
      region: ${aws_region}
  secretsManager:
    type: awsSecretManager
    awsSecretManager:
      authenticationType: instanceProfile
      region: ${aws_region}
      kms: ${kms_key_arn}
serviceAccount:
  annotations:
    <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: ${role_arn}```
7

keys in there are authenticationType to instanceProfile and then setting the service account role ARN annotation so that the pods are launched with a service account. instanceProfile doesn’t mean only EC2 instance profile, it just means use the default AWS credential provider chain, as implemented in the SDKs, which will pull in standard EKS IRSA roles

8

instanceProfile is confusing T_T

9

I would have never gotten that. Thank you for saving me 3 cups of coffee.

10

<@U03FNJGED7D> got it working thanks again for the help!