Summary
Issue with storing connector secrets in AWS Secrets Manager instead of database in Airbyte on AWS EKS
Question
Hello,
We have setup Airbyte on AWS EKS and use external Postgres. We would like to use AWS Secrets Manager to store connector secrets instead of storing in db. We have followed https://docs.airbyte.com/deploying-airbyte/integrations/secrets#secrets but it is still storing secrets in database. The secrets manager is setup using instance-profile. Would like to hear from other members of the group if they had experienced same issue. Thank you.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["aws-secrets-manager", "airbyte-platform", "aws-eks", "external-postgres", "connector-secrets"]
Did you setup the values.yaml correctly? I’m using Google’s secret manager and for me it’s working well.
You were right. Had a wrong indentation. Thanks!
Had one question for you, I am seeing airbyte creating random secrets, which can grow over time and become difficult to manage. Is there any way that the airbyte secrets can be created inside a single parent secret?
<@U070S8Q52QM> I’m not sure, I just got my instance running a few days ago. That being said, I was just thinking that this could be a problem. One solution I though could work was to create a separate project in GCP for Airbyte secrets, that way they don’t crowd out the main project.