Trouble applying external database on Airbyte running on GKE

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

User is facing difficulties applying an external database to Airbyte running on GKE. Error related to timeout occurs when trying to upgrade the installation with specific values.


Question

Hello Folks, I’m having difficult to apply the external database on airbyte.
My airbyte is running under GKE and I already setup the postgres database, but when I try to upgrade the installation looking to values, they return error related to timeout.
Below are the command to upgrade the airbyte and attached is my values file.

Please, someone can help me with this?

  --values values.yaml \
  --set ingress.enabled=true \
  --set ingress.annotations."kubernetes\.io/ingress\.class"=nginx \
  --set ingress.annotations."cert-manager\.io/cluster-issuer"="letsencrypt-prod" \
  --set ingress.tls[0].hosts[0]="host" \
  --set ingress.tls[0].secretName="airbyte-tls"```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1713378779731049) if you want to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["external-database", "airbyte", "GKE", "timeout", "upgrade", "values-file"]
</sub>
2

<@U06TNRDCA2E> Are you using Google Cloud SQL or another postgres provider?

And are you creating the secret on the command line via kubectl or another way?

3

Hello <@U035912NS77> Yes, I’m using Google Cloud SQL and create the secret file direct on kubectl.

4

<@U06TNRDCA2E> Here’s what works for me on our GKE Autopilot cluster:
• host is the private/internal IP of Cloud SQL
• I would put Airbyte in its own DB, which you’ll want to create through the Cloud SQL UI in advance (the default name is db-airbyte) and update the database value accordingly
• jdbcUrl is in the format "jdbc:postgresql://&lt;IP&gt;:&lt;PORT&gt;/&lt;DB&gt;" or using the example above this would look something like "jdbc:<postgresql://10.123.123.123:5432/db-airbyte>"
Again, that’s up and running for me but took some fiddling to make it happy with the format of the jdbcUrl. Hope it helps!

5

<@U035912NS77> Tks so much for your help. I’ll test this now in my environment and check.

6

Tks <@U035912NS77> appears that works…they accept the airbyte update looking to the values file. I change as you mentioned.

But when I looking to the postgres I don’t see any table created yet. I know if they need some time to create this?

Again, tks so much for your help!

7

I would list the database and see if they create another one. I know Temporal creates temporal and temporal_visibility, but I haven’t tried connecting with passing the database as postgres, so I’m not sure if they only use that as the connection and then switch or if they actually create tables there.

8

Tks a lot Justin.
I will update the files and try to use the Google Load Balancer, probably will be more easier that nginx.
And also update the vaules.yaml

Tks again

9

Hello <@U035912NS77> sorry back again on this.
I check my postgres and airbyte don’t create any table there. I already checked the network communication between the cluster and database, its works fine.
Do you know if there are any other configuration or something needed? I also checked that on airbyte deploy was created the airbyte-db services. But don’t know if this is related to internal database or if there are something inside this pod, that needed changed.

10

I think the issue is actually with your values.yaml—all those keys (global, postgresql, and externalDatabase) should be at the top level, not under another object like they are now (right now, you have them under airbyte). Right now, I’m guessing it’s ignoring all that config completely.

I would also recommend creating a specific DB for Airbyte in the Cloud SQL instance settings—the default is db-airbyte. This will make it easier to see if it works, but note that you need to create this before connecting (since the connection relies on the DB). And if you change it, update the JDBC URL with that as well.

I personally prefer to also put everything into a namespace so that I can completely tear it down and be sure that I’m not inheriting messy config from my previous attempts (which were many :joy:). You can do that with something like—basically everything just needs the --namespace argument:

kubectl create namespace airbyte-ns
kubectl create secret generic db-secrets --from-literal=DATABASE_PASSWORD=your-super-secret-password --namespace=airbyte-ns
helm install --values values.yaml airbyte airbyte/airbyte --namespace=airbyte-ns

# TO UPGRADE IN-PLACE WITH NEW VALUES
helm upgrade --values values.yaml airbyte airbyte/airbyte --namespace=airbyte-ns

# TO WIPE THE INSTALL COMPLETELY
helm uninstall airbyte --namespace=airbyte-ns
# (wait) Now Wait until you see all the resources deleted in GKE, especially any load balancer it creates
# WARNING: This next line will delete everything in the namespace, including your secrets and anything you've added via helm apply, and get you to clean state to make another attempt
kubectl delete namespace airbyte-ns
# (wait) Note that this can take a couple minutes```
Another advantage of a namespace is that you can filter to it in all the GKE stuff, so you can see _just_ your Airbyte deployment and not any of the system-level stuff. By default the namespace is implicitly `default`, but you could have other things in there as well which is why I like to specify one explicitly.

I also noticed from your previous screenshot, you're passing `ingress` values with `--set`, but the path is wrong . . . these should be `webapp.ingress.*`, not `ingress.*`. On that end, I'd also recommend letting it provision a native Google HTTPS External LB. Here's what's worked for me with that:
```webapp:
  service:
    annotations:
      # This creates a Network Endpoint Group, which allows you to use a ClusterIP service as a backend
      <http://cloud.google.com/neg|cloud.google.com/neg>: '{"ingress": true}'

  ingress:
    enabled: true
    annotations:
      <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: gce
      <http://kubernetes.io/ingress.global-static-ip-name|kubernetes.io/ingress.global-static-ip-name>: &lt;your-already-created-static-ip-reservation-name&gt;
      <http://networking.gke.io/managed-certificates|networking.gke.io/managed-certificates>: &lt;your-already-created-managed-cert-name&gt;
      # You can also use pre-shared certs and such here with different annotations
    hosts:
      - host: <your-host-name.example.com>
        paths:
          - path: /*
            pathType: ImplementationSpecific```
Again, just a preference, but also makes it play nicer in things like shared-VPC environments and sticks with the more Google-native functionality. (The only exception I would make and would prefer an nginx frontend is if I need URL Rewrite logic, which is unlikely in a deployment like this.)