Managing AWS secrets in Airbyte

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

How to determine which secrets are in use and which can be deleted in Airbyte with AWS secret manager


Question

Hi everyone :fire:
My company has Airbyte in production for some time and we have accumulated thousands of secrets in AWS secret manager (big $:heavy_dollar_sign:$$).
I can’t find a way to determine which secrets are in use and which can be deleted :disappointed:
Any pointers / guidance will be appreciated.
:pray:


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["aws-secret-manager", "secrets-management", "airbyte-platform"]
2

Hi, as of 0.63.10 Airbyte now correctly sweeps secrets. Secrets were previously left behind even as connectors configuration were upgraded/deleted. Upgrading to this version should stop things from getting worse.

First, how many active sources/destinations do you have? Each source/destination correspond to at least one secret so if you have a ton, it’s expected to some amount of secrets around.

3

Thanks for the quick response!
We are running Airbyte 0.42.0 so there is hope :wink:

4

I will get back to you with active source/destination count.

5

if you have a few active sources/destination, sweeping the old secrets is much easier. If you don’t… it’ll be a bit more involved, however I’m happy to give you some pointers

6

we have more that 100 connections in production and don’t know how the upgrade from 0.42.0 to 0.63.10 will impact these connections, so any pointers will help till we figure out how to upgrade our Airbyte version.

7

Got it.

Let’s start by figuring out how many active secrets you have:

  1. Download all active actors from the database by running this query select configuration from actor where tombstone = false; An actor is a Source of Destination.
  2. After that run grep -oE 'airbyte_(oauth_)?workspace_[0-9a-f-]+_secret_[0-9a-f-]+_v[0-9]+' <file_name> on the downloaded file. This should give you all the active secrets per line.
    The above 2 steps should give us a good sense of how many secrets we need to delete, and how much money is to be saved.
8 
grep -oE 'airbyte_(oauth_)?workspace_[0-9a-f-]+_secret_[0-9a-f-]+_v[0-9]+'   actor.configuration |wc -l
     726```
9

Staging:

     608```
10

dev:

    1645```
11

Note: My actor db select is without “where tombstone = false;”
Is this critical?

12

Can you share the breaking changes between 0.42.0 and 0.63.10?

13

Have a safe and happy weekend
Thanks for all your help!!!

14

Great. What I would do now is iterate through the current secrets and delete those that do not result from the grep statement (assuming Airbyte is the only one using the secret store). You’d probably have to write some scripts for this.

> Note: My actor db select is without “where tombstone = false;”
This depends on how you are deleting. This returns active secrets which works if you are looking to delete everything else in the secret store except the results. Querying for tombstone = true would return all the secrets you need to delete.

> Can you share the breaking changes between 0.42.0 and 0.63.10?
There are too many to list… Your instance is far behind, so I’d suggest putting aside a day or so and upgrading by minor versions. We know we need to be better with upgrade paths, and we are working on having more official guidance in a few months or so

15

I started deleting secrets based on your guidance, I will update as I progress.