Trouble installing Airbyte locally with abctl command

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

The user is facing an error while trying to install Airbyte locally using the abctl command. The error message indicates a failure in installing the Airbyte Helm Chart.


Question

i’m attempting to install with abctl with this command

    --secret db-secrets.yaml \
    --secret auth-secrets.yaml \
    --values values.yaml```
and currently getting this:
```  INFO    Using Kubernetes provider:
            Provider: kind
            Kubeconfig: /home/airbyte/.airbyte/abctl/abctl.kubeconfig
            Context: kind-airbyte-abctl
 SUCCESS  Found Docker installation: version 24.0.5                                                                                                                                   
  INFO    No existing cluster found, cluster 'airbyte-abctl' will be created                                                                                                          
 SUCCESS  Port 8000 appears to be available                                                                                                                                           
 SUCCESS  Cluster 'airbyte-abctl' created                                                                                                                                             
  INFO    Namespace 'airbyte-abctl' created                                                                                                                                           
  INFO    Persistent volume 'airbyte-minio-pv' created                                                                                                                                
  INFO    Persistent volume 'airbyte-volume-db' created                                                                                                                               
  INFO    Persistent volume claim 'airbyte-minio-pv-claim-airbyte-minio-0' created                                                                                                    
  INFO    Persistent volume claim 'airbyte-volume-db-airbyte-db-0' created                                                                                                            
 SUCCESS  Secret from '/home/airbyte/db-secrets.yaml' created or updated                                                                                                              
 SUCCESS  Secret from '/home/airbyte/auth-secrets.yaml' created or updated                                                                                                            
  INFO    Starting Helm Chart installation of 'airbyte/airbyte' (version: 0.594.0)                                                                                                    
  ERROR   Failed to install airbyte/airbyte Helm Chart                                                                                                                                
  ERROR   Unable to install Airbyte locally                                                                                                                                           
  ERROR   unable to install airbyte chart:```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1726517964240249) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["install", "abctl", "local", "error", "helm-chart", "airbyte"]
</sub>
2

abctl uses kind cluster (https://kind.sigs.k8s.io/) under the hood
If you want to debug, here’s an instruction

install kubectl (https://kubernetes.io/docs/tasks/tools/#kubectl) and kubectx + kubens (https://github.com/ahmetb/kubectx)
install also k9s (https://k9scli.io/)

then execute commands:

KUBECONFIG=~/.airbyte/abctl/abctl.kubeconfig kubectl config view --flatten &gt; ~/.kube/config
kubectx kind-airbyte-abctl
kubens airbyte-abctl```
with `k9s` you can check details about pods and logs
3

btw, kubectl exists in the container, so I find it useful to run, for example:
docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl get pods

4

I also found today that if you install kind, you can run
kind export logs --name=airbyte-abctl ./logsdir
and that will give you all sorts of pod/container logs

5

I’d really like to know where your install failed, because I’ve been trying to debug these failures and it’s hard to tell what exactly unable to install airbyte chart means

6

getting No resources found in airbyte-abctl namespace. from your first command, and I’m not sure what to even look for in the logsdir :joy: i have zero kubernetes experience

7

can you run your install command again with --verbose and send the output when it’s done?

8

Does it fail quickly? Or does it take awhile?

9

it fails immediately after successfully creating the cluster, which takes about a minute. here’s the <https://pastebin.com/YxHT0wbE|verbose output>

10

thanks. weird. I might have accidentally hidden an error in v0.16.0. I’ve just released https://github.com/airbytehq/abctl/releases/tag/v0.16.1|v0.16.1 which might help

11

thanks for the quick update. Running right now

12

ok getting: unable to install airbyte chart: unable to install helm: execution error at (airbyte/templates/env-configmap.yaml:34:6): You must set global.database.host when using an external database

13

ok, ya, sorry about that hidden error. looks like you have a missing value maybe

14

got it, will try to troubleshoot this on my own for now thank you!

15

i seemed to have gotten it working. my values.yaml looked like this:

  enabled: false

global:
  database:
    secretName: "airbyte-config-secrets"
    hostSecretKey: "database-host"
    portSecretKey: "database-port"
    databaseSecretKey: "database-name" 
    userSecretKey: "database-user"
    passwordSecretKey: "database-password"```
and I set the actual credentials in db-secrets.yaml
```apiVersion: v1
kind: Secret
metadata:
  name: airbyte-config-secrets
type: Opaque
stringData:
  # Database Secrets
  database-host: <http://dummyrds.us-west-2.rds.amazonaws.com|dummyrds.us-west-2.rds.amazonaws.com>
  database-port: 5432
  database-name: postgres
  database-user: airbyte_admin
  database-password: testtesttest```
I had to actually fill in the db credentials in values.yaml (except for the actual password) for me to get past the error
16

which i think is a bug? should I file?

17

hm. I’m not 100% sure, but I don’t think it’s a bug. I think only the password is a secret in the current chart setup

18

<@U05N5APSYBE> what postgres version do you used?
I think for RDS postgets only work on version 13.x, above that you need to turned of the force_ssl. this is based on my experienced.

19

I’m using 15, which seems to work fine

20

are there other configs i can add to the values.yaml that apply for the abctl deployment as it does the kubernetes deployment? I seem to only run 4 syncs concurrently and want to know how i can scale. the only info i found in the docs are about MAX_CHECK_WORKERS and MAX_SYNC_WORKERS for <https://docs.airbyte.com/operator-guides/configuring-airbyte|kubernetes deployments> not for abctl deployments