Issue with creating source or destination using external Postgres DB

Slack Knowledge Base Q&A Connector Questions
, , , ,
21

The postgres database is in AWS RDS. Are you running Airbyte OSS in Kubernetes? If not then how we set it up might not be super relevant

22

It still might be. The issue here is the database connection itself

23

This is what we used for our helm values:

    enabled: false
  externalDatabase:
    user: root
    database: postgres
    port: 5432
    existingSecret: "airbyte-1p-airbyte-db"
    existingSecretPasswordKey: "password"```
24

And then we set externalDatabase.host for each environment

25

Thanks for sharing! I guess the issue might be with the gcp postgres

26

Are you able to connect to the database with the user you setup for airbyte at all? from a database client?

27

Yes, I can even see the records in the databases created by airbyte and temporal

28

Database permissions are setup properly?

29

I tried setting it up with the admin user to make sure

30

It was still the same

31

Do you have query logging? Can you see which specific queries are failing?

32

That’s the issue. No query is failing. It’s just a bunch of airbyte-temporal issues

33

It started to happen after I migrated from v0.50.7

34

We have a running document of issues with Airbyte that we’ve had to fix

35

Have you tried dropping the temporal database from postgres since the upgrade?

36

I tried setting up from scratch with a newly created db

37

We’re using an external Google Cloud SQL postgres instance from a private GKE cluster without issues. Here’s the example config you can probably merge with yours:

  database:
    secretName: &db-secret <your-kubernetes-secret-name>
    secretValue: &db-secret-key <your-kubernetes-secret-key>

postgresql:
  enabled: false

externalDatabase:
  host: "<your-postgres-server-internal-ip-address>"
  user: <your-postgres-db-username>
  password: ""
  existingSecret: *db-secret
  existingSecretPasswordKey: *db-secret-key
  database: db-airbyte
  port: 5432
  jdbcUrl: "jdbc:postgresql://<your-postgres-server-internal-ip-address>:5432/db-airbyte"```
(you can also change the `database` name if needed, but make sure you also change it in `jdbcUrl`)

It's also worth noting that setting `postgresql.enabled` to `false` just disables the creation of the `airbyte-db` pod—none of the other settings in that section are used when you're using an external postgres DB.