Pre-upgrade hook failure in Airbyte Helm Chart installation

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

The user is facing a pre-upgrade hook failure while installing the Airbyte Helm Chart. They are seeking guidance on where to find an explanation for the failure and how to enable sufficient diagnostics in the Helm chart.


Question

I am trying to install with abctl v0.14.1 and the default 0.524.0 chart

I have custom --secret file and a custom --values file

When I run with -v I get this:

DEBUG   Successfully assigned airbyte-abctl/airbyte-abctl-airbyte-bootloader to airbyte-abctl-control-plane
  DEBUG   helm - DEBUG: Add/Modify event for airbyte-abctl-airbyte-bootloader: MODIFIED
  DEBUG   helm - DEBUG: Pod airbyte-abctl-airbyte-bootloader pending
  DEBUG   Container image "airbyte/bootloader:0.64.1" already present on machine
  DEBUG   Created container airbyte-bootloader-container
  DEBUG   Started container airbyte-bootloader-container
  DEBUG   helm - DEBUG: Add/Modify event for airbyte-abctl-airbyte-bootloader: MODIFIED
  DEBUG   helm - DEBUG: Pod airbyte-abctl-airbyte-bootloader running
  DEBUG   helm - DEBUG: Add/Modify event for airbyte-abctl-airbyte-bootloader: MODIFIED
  DEBUG   helm - DEBUG: Pod airbyte-abctl-airbyte-bootloader running
  DEBUG   helm - DEBUG: Add/Modify event for airbyte-abctl-airbyte-bootloader: MODIFIED
  DEBUG   helm - DEBUG: warning: Upgrade "airbyte-abctl" failed: pre-upgrade hooks failed: 1 error occurred:
                * pod airbyte-abctl-airbyte-bootloader failed
  DEBUG   helm - DEBUG: release upgrade failed: pre-upgrade hooks failed: 1 error occurred:
                * pod airbyte-abctl-airbyte-bootloader failed
  ERROR   Failed to install airbyte/airbyte Helm Chart
  ERROR   Unable to install Airbyte locally
  ERROR   unable to install airbyte chart: unable to install helm: pre-upgrade hooks failed: 1 error occurred:
                * pod airbyte-abctl-airbyte-bootloader failed%           ```
which gives me a fact “pre-upgrade hooks” failed but fails to provide any explanation as to why the pre-ugrade hooks failed

I am running on a freshly-minted Amazon Linux 2023.

This was working yesterday (with v0.13.1) but has failed everytime I have tried it today.

Where do I look to find an explanation for why the pre-upgrade hook failed. What do i do to build helm chart that has sufficient diagnostics enabled to allow me to understand this failure?

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1725415232566839) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["pre-upgrade-hook-failure", "airbyte-helm-chart", "diagnostics", "helm-chart", "installation"]
</sub>
2

I’m curious about this, because I’m trying to debug why these bootloader + helm hook failures occur, and I’ve had a hard time reproducing the issue.

I’m not sure how best to start debugging. Perhaps you could try to inspect the airbyte-abctl-airbyte-bootloader pod? Something like

[ec2-user@ip-172-31-20-234 ~]$ kubectl -n airbyte-abctl logs airbyte-abctl-airbyte-bootloader```
You'll need to install `kubectl`: <https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-using-native-package-management>
3

ok, thanks for that.

I discovered the issue in my case, I had mistakenly encoded secrets in my secrets.yaml file in base64. When I undid that encoding, the datasource was able to connect as expected.

4

ok thanks, it’s helpful to know these errors can be caused by secrets.yaml

5

strictly speaking, it wasn’t secrets.yml - technically it was a bad database user, that was causing the connections to the airbyte database to blow up.

6

I understand abctl is aiming to hide the complexities of helm charts and kubectl from users, the reality is that you can’t effectively diagnose a broken airbyte-abctl install/configuration without a good working knowledge of the various tools such as kubectl and various k8s concepts like pods, deployments, confligmaps, secrets, charts etc

an abctl-specific troubleshooting guide that doesn’t assume extensive pre-existing knowledge of the underlying tools like kubectl would be really, really good, I think.

7

I’m pondering a command like abctl local logs or something that could grab logs from internal pods for you. And, perhaps we can recognize bootloader errors and automatically extract the logs during install