Summary
Unable to add custom connectors created by CDK in certain environments like abctl and Kubernetes using Kind or minikube. Looking for reasons why connectors cannot be added and guidance on the type of Kubernetes cluster suitable for adding connectors on EC2 (Linux). Able to add connectors created by CDK using Kubernetes with docker desktop.
Question
Hi, I could not add custom connectors created by CDK under following enviroments. What is the reason why I could not add connectors ? What kind of Kubernetes cluster should I use to add connectors on EC2(Linux)?
Environments where I could not add connectors created by CDK
• abctl
• kubernetes using Kind or minikube
I could add connectors created by CDK using kubernetes using docker desktop.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["custom-connectors", "cdk", "environments", "abctl", "kubernetes", "kind", "minikube", "ec2-linux", "docker-desktop"]
Where do you store Docker images for your custom connectors? Locally in Docker, private or public registry?
I created Docker images of custom connectors created using CDK and airbyte-ci on local PC.
When I used airbyte on local PC and the local Docker images, I could add the connectors using Kubernetes on docker desktop or Docker (not Kubernetes). But I could not add them using abctl or Kubernetes on Kind or minikube.
When I used airbyte on EC2 and the Docker images which I created pulled from ECR, I could add the connectors using Docker (not Kubernetes). But I could not add them using abctl or Kubernetes on Kind.
abctl uses Kind cluster under the hood
I was able to reproduce this issue locally. At this moment, the easiest solution I found is to load Docker image with command
kind load docker-image airbyte/custom-connector:dev --name airbyte-abctl
<@U01MMSDJGC9> do you think that airbyte-ci could have a task that tries to load Docker image into Kind cluster provisioned by abctl when airbyte-abctl Kind cluster is detected? Or maybe it should be a responsibility of abctl command? 
Also, it would be nice to update docs https://docs.airbyte.com/integrations/custom-connectors
Thanks. The command worked!
Good question. I’ll raise this to the tooling team. For now let’s update the docs!