Trouble installing Airbyte locally with abctl command

Summary

The user is facing an error while trying to install Airbyte locally using the abctl command. The error message indicates a failure in installing the Airbyte Helm Chart.


Question

i’m attempting to install with abctl with this command

    --secret db-secrets.yaml \
    --secret auth-secrets.yaml \
    --values values.yaml```
and currently getting this:
```  INFO    Using Kubernetes provider:
            Provider: kind
            Kubeconfig: /home/airbyte/.airbyte/abctl/abctl.kubeconfig
            Context: kind-airbyte-abctl
 SUCCESS  Found Docker installation: version 24.0.5                                                                                                                                   
  INFO    No existing cluster found, cluster 'airbyte-abctl' will be created                                                                                                          
 SUCCESS  Port 8000 appears to be available                                                                                                                                           
 SUCCESS  Cluster 'airbyte-abctl' created                                                                                                                                             
  INFO    Namespace 'airbyte-abctl' created                                                                                                                                           
  INFO    Persistent volume 'airbyte-minio-pv' created                                                                                                                                
  INFO    Persistent volume 'airbyte-volume-db' created                                                                                                                               
  INFO    Persistent volume claim 'airbyte-minio-pv-claim-airbyte-minio-0' created                                                                                                    
  INFO    Persistent volume claim 'airbyte-volume-db-airbyte-db-0' created                                                                                                            
 SUCCESS  Secret from '/home/airbyte/db-secrets.yaml' created or updated                                                                                                              
 SUCCESS  Secret from '/home/airbyte/auth-secrets.yaml' created or updated                                                                                                            
  INFO    Starting Helm Chart installation of 'airbyte/airbyte' (version: 0.594.0)                                                                                                    
  ERROR   Failed to install airbyte/airbyte Helm Chart                                                                                                                                
  ERROR   Unable to install Airbyte locally                                                                                                                                           
  ERROR   unable to install airbyte chart:```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1726517964240249) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["install", "abctl", "local", "error", "helm-chart", "airbyte"]
</sub>

abctl uses kind cluster (https://kind.sigs.k8s.io/) under the hood
If you want to debug, here’s an instruction

install kubectl (https://kubernetes.io/docs/tasks/tools/#kubectl) and kubectx + kubens (https://github.com/ahmetb/kubectx)
install also k9s (https://k9scli.io/)

then execute commands:

KUBECONFIG=~/.airbyte/abctl/abctl.kubeconfig kubectl config view --flatten &gt; ~/.kube/config
kubectx kind-airbyte-abctl
kubens airbyte-abctl```
with `k9s` you can check details about pods and logs

btw, kubectl exists in the container, so I find it useful to run, for example:
docker exec -it airbyte-abctl-control-plane kubectl -n airbyte-abctl get pods

I also found today that if you install kind, you can run
kind export logs --name=airbyte-abctl ./logsdir
and that will give you all sorts of pod/container logs

I’d really like to know where your install failed, because I’ve been trying to debug these failures and it’s hard to tell what exactly unable to install airbyte chart means

getting No resources found in airbyte-abctl namespace. from your first command, and I’m not sure what to even look for in the logsdir :joy: i have zero kubernetes experience

can you run your install command again with --verbose and send the output when it’s done?

Does it fail quickly? Or does it take awhile?

it fails immediately after successfully creating the cluster, which takes about a minute. here’s the <https://pastebin.com/YxHT0wbE|verbose output>

thanks. weird. I might have accidentally hidden an error in v0.16.0. I’ve just released https://github.com/airbytehq/abctl/releases/tag/v0.16.1|v0.16.1 which might help

thanks for the quick update. Running right now

ok getting: unable to install airbyte chart: unable to install helm: execution error at (airbyte/templates/env-configmap.yaml:34:6): You must set global.database.host when using an external database

ok, ya, sorry about that hidden error. looks like you have a missing value maybe

got it, will try to troubleshoot this on my own for now thank you!

i seemed to have gotten it working. my values.yaml looked like this:

  enabled: false

global:
  database:
    secretName: "airbyte-config-secrets"
    hostSecretKey: "database-host"
    portSecretKey: "database-port"
    databaseSecretKey: "database-name" 
    userSecretKey: "database-user"
    passwordSecretKey: "database-password"```
and I set the actual credentials in db-secrets.yaml
```apiVersion: v1
kind: Secret
metadata:
  name: airbyte-config-secrets
type: Opaque
stringData:
  # Database Secrets
  database-host: <http://dummyrds.us-west-2.rds.amazonaws.com|dummyrds.us-west-2.rds.amazonaws.com>
  database-port: 5432
  database-name: postgres
  database-user: airbyte_admin
  database-password: testtesttest```
I had to actually fill in the db credentials in values.yaml (except for the actual password) for me to get past the error

which i think is a bug? should I file?

hm. I’m not 100% sure, but I don’t think it’s a bug. I think only the password is a secret in the current chart setup

<@U05N5APSYBE> what postgres version do you used?
I think for RDS postgets only work on version 13.x, above that you need to turned of the force_ssl. this is based on my experienced.

I’m using 15, which seems to work fine

are there other configs i can add to the values.yaml that apply for the abctl deployment as it does the kubernetes deployment? I seem to only run 4 syncs concurrently and want to know how i can scale. the only info i found in the docs are about MAX_CHECK_WORKERS and MAX_SYNC_WORKERS for <https://docs.airbyte.com/operator-guides/configuring-airbyte|kubernetes deployments> not for abctl deployments