Trouble migrating from docker compose to abctl

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

User is facing issues migrating from docker compose to abctl, encountering 404 not found error and timeout error. User is concerned about the 1 Aug deadline for docker compose deprecation.


Question

Hi everyone. I’ve been trying to migrate from docker compose to abctl the past few days but haven’t had luck. I’ve tried in version 0.9.1 and 0.9.2, and with commands abctl local install --migrate and abctl local install --migrate --host <my-host-name>, but still ends up with 404 not found error from nginx. Log wise, everything was alight for the first command, while the second command returns a timeout error about waiting for ingress and failing to install Airbyte locally. Will 1 Aug be a hard stop for the docker compose deprecation? What happens if I still haven’t managed to migrate to abctl by then?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["migrate", "docker-compose", "abctl", "404-not-found", "timeout-error", "nginx", "deprecation"]
2

I assume that docker-compose.yaml, run-ab-platform.sh might be gone from repositories and setup with docker compose won’t be possible any longer. You still should be able to use docker compose without any issues if you keep docker-compose.yaml, .env and other files. Over time, less and less support will be available from community and you’ll be on your own with that kind of setup.

Have you tried to debug 404 not found error from nginx? Do you need some help with it?

3

Yes please, I would appreciate it if we can debug the 404 not found error, else I would need to create a whole new Airbyte instance and set up all 250+ connections from scratch again, which would take quite some time.

4

abctl uses kind (https://kind.sigs.k8s.io/) under the hood

install kubectl (https://kubernetes.io/docs/tasks/tools/#kubectl) and kubectx + kubens (https://github.com/ahmetb/kubectx)
install also k9s (https://k9scli.io/)

then execute commands:

KUBECONFIG=~/.airbyte/abctl/abctl.kubeconfig kubectl config view --flatten > ~/.kube/config
kubectx kind-airbyte-abctl
kubens airbyte-abctl```
5

what output do you get for abctl local status?

6

Hi. I just tried installing version 0.11.0 today, and still get 404 error. This is the output I get after running abctl local status.

7

Oh do I have to run these commands that you previously mentioned before following the migration guide?

KUBECONFIG=~/.airbyte/abctl/abctl.kubeconfig kubectl config view --flatten > ~/.kube/config
kubectx kind-airbyte-abctl
kubens airbyte-abctl```
I followed the official migration guide exactly, which is these three commands. I didn't run anything else.
8

Please run those commands before running migration

9

Another question, what output do you get for docker compose ps ?

Today I upgraded abctl and I performed abctl local install that used the latest helm chart version
I had Airbyte already installed, so it tried to perform upgrade.
I had some issue with nginx, because new pod tried to use the same as an old nginx pod

I suspect that this might the case for you - port is already used (probably by airbyte-proxy in docker-compose.yaml) and nginx cannot use it
Can you try to add option --port 8888 to any of your abctl local install --migrate ... commands?
You can check other ports if 8888 doesn’t work for you.
After successful migration you should be able to run abctl local install later and switch back to default port 8000.