Summary
When trying to upload a custom Python connector docker image on Airbyte UI, encountering ‘Internal Server Error: Get Spec job failed’ despite successful local spec job run.
Question
Hi all, I developed a custom Airbyte Python http based connector. I built my docker image and I am trying to upload it on Airbyte UI and getting this error: “Internal Server Error: Get Spec job failed.” I cannot see any other logs, when I run the spec job on my local environment it runs successfully, what is the problem here?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["custom-python-connector", "docker-image", "airbyte-ui", "internal-server-error", "get-spec-job"]
Try to check the logs of the worker. I had the same problem and usually the worker logs will give you the real reason for the import error. The first problem I has was related to Airbyte not being able to pull the image from the container registry.
Hi Mert, a common issue is that you might be building your docker image on an architecture that isn’t the one you’re pulling it from. E.g. if you are building it on an ARM machine and pulling it from an AMD-based pod. You’ll notice that all of our official images are published under multiple architectures.
This might not be the problem, but its a common one! Definitely follow <@U0668M7G22W>’s advice too. It could also have to do with where you pushed the image to and if your cluster has access to pull from it
hey hi, I am also trying to build airbyte python based connectors but i have trobule in doing that can you help me?
Hi <@U074QP7HV5X>, let me know what is your problem and I can try to share what I know about it
Hi <@U0668M7G22W>, I want to use postgres as my source but i did not find postgres source in pyaitbyte? so is there any other way to solve the problem?