All connection tests fail after importing local configuration into k8s deployment

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Debian
  • Deployment: Kubernetes
  • Airbyte Version: v0.40.0-alpha
  • Source name/version: all
  • Destination name/version: all
  • Step: testing connections after importing a configuration
  • Description: I’m in the process of upgrading from 0.35.0 to 0.40.0 and, as part of this, I’m trying to import a configuration from a local environment running with docker-compose, into a test environment running in k8s. I’m able to import the configuration, but all connector tests fail after doing so.

Steps to replicate:

  • docker compose up a local environment running v0.40.0
  • Add a connector (any connector will do) and make sure the tests succeed
  • Export the configuration; import it into a k8s environment also running v0.40.0
  • Run the tests for the connector (they should fail)
  • Update the credentials for the connector in the UI for the k8s environment (they should still fail)
  • Create the same connector with the same credentials in the k8s environment directly (they should succeed)

The last part is what really confuses me: the same connector, in the same environment, with the same credentials, works if I create it in k8s directly, but does not work if it was part of the config I imported. Somehow, it seems that the connector is broken when I import the config, and broken in a way that updating the credentials for the connector does not fix it.

Another hint is the error messages I get: 401 Client Error: Unauthorized for url for BambooHR, invalid access key ID (for S3), a 403 for Amplitude… all of which point to the secret not being passed along correctly. But this might be a red herring.

Any help would be appreciated - until I get this to work, I’m unable to upgrade our prod environment :frowning:

Oliver

Hey it would be of more help if you can share the logs of the failure sync otherwise I would suggest

  1. Upgrade the current airbyte docker to 0.40.0
  2. export the config
  3. Import it back again in k8s