Summary
Inquiring about defining persistent storage for webapp pods in Helm charts to ensure updates made in the UI survive pod restarts.
Question
When deploying the Helm charts, is it possible to define persistent storage for the webapp pods? In the UI you can trigger it to download the latest versions of destinations and sources. Seems like this update does not survive pod updates. I’m wondering if I add persistent volume to the pod would that get used and the updates I do in the UI survive pod restarts.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["persistent-storage", "webapp-pods", "helm-charts", "ui-updates", "pod-restarts"]
You are referring to those versions?
Let me describe how it works for me.
When I change anything on that dashboard, web browser sends request to /api/v1/source_definitions/update endpoint and record is changed (column default_version_id) in actor_definition table in Airbyte’s database. Change is persistent.
When I upgrade Airbyte I might see some upgrades for source/destinations connectors that I haven’t used at all, but for connectors that are in use I haven’t seen any changes. This behavior is important in my project, because some connectors are purposefully downgraded to a specific version.
I don’t remember, there could have been a bug long time ago, but I haven’t experienced any issues related to that as long as we started to use Airbyte in my project.
<@U05JENRCF7C> thanks, so clicking the “Upgrade all” doesn’t actually download anything? When I think about it, that does make sense. I’m still tuning my instance (changing resource request amounts, etc) and every time I do that the updates in the UI I’ve ran before seem to get reversed. Meaning when I access the webapp next time, I see those red dots prompting me to upgrade them. Not sure if the bootloader does something in the DB that resets the tables. None of the connections I’ve created get wiped out though, so it’s not doing a complete “factory reset” into the database.
I’m using Cloud SQL for the database so the DB stays constant from update to update.
“Upgrade all” used to download Docker images, but it was changed some time ago. Think about pulling hundreds of Docker images. I’m glad that was changed.
I recommend to stop paying any attention to upgrading connectors you don’t use. With red dots we are kind of tricked to perform those upgrades. It is UI/UX design and I would rather have only notifications for connectors that I use.
When it comes to bootloader, it performs some database migrations and I think I saw something related to fetching information about connectors from registry. More on that, for sure, can be found somewhere in repositories airbyte and/or airbyte-platform
Yeah, the red dots bother me at a “deep level” and I’m just compelled to upgrade them all 
But yeah, if it just updates DB values then it’s not a big deal to redo it.
Yeah, I know, but new versions of connectors and new connectors are released very often so it’s never-ending story with this red dot.
If you don’t want to see the red dot, you might consider blocking/hiding it with browser extension, e.g. Adblock Plus
Makes sense, thanks for clarifying these things <@U05JENRCF7C>