Error with api/v1/source_definitions/create when running on GCP

Connector Questions & Issues
1
  • Is this your first time deploying Airbyte?: Yes
  • OS Version / Instance: Linux(GCP compute engine)
  • Memory / Disk: you can use something like 4Gb / 1 Tb
  • Deployment: Docker
  • Airbyte Version: 0.39.5-alpha
  • Source name/version: Docker Hub
  • Destination name/version:
  • Step: The issue is happening during importing the connector
  • Description:
    Hi, I’m facing an issue while importing an home-built connector to an airbyte instance running on GCP compute engine.
    Once I clicked on the add button, I get a 500 error:
    java.lang.IllegalStateException: Get Spec job failed.

I trouble shooted the issue on my side:

  • CE instance can connect to docker hub as it’s working with other containers
  • I can import the mentionned container on airbyte running locally (:sweat_smile:)
  • The API call is not returning more information than the message added to the description
  • I’m exploring the possibility of a blocked process by the firewall …

Do you have any idea of what’s happening ? Or an idea to restrict the search scope ?

Thanks a lot for your help,

2

The issue could be due to the OS/Arch (linux/arm64/v8) of this image.
It would explain why it’s working locally and not remotely

3

@alexisc could you try to run docker run connector-image:tag spec on your GCP VM and share the container output?
It might indeed be an architecture-related problem.

4

It’s confirmed it was due to an image built on a Apple M1 not running properly on the VM.
It will probably be interresting to display a more explicit error than Get Spec job failed.
Display the docker run error ?

This is not an issue with airbyte. Sorry for the noise!
Thanks for the support though

5

@alexisc you might want to use Docker Buildx to create a multi architecture image for your connector.
And you are right that the error message should be more explicit!

6

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.