Issue with Google Artifact Registry and Image Pulling in Airbyte

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

Airbyte is failing to pull images from a private registry (Google Artifact Registry) when adding a custom connector. The user is running Airbyte in Docker Compose and manually pulling the image allows Airbyte to find and add it. The question is whether Airbyte should be able to pull the image from the registry on its own.


Question

Hello, I’m experiencing issues with Google Artifact Registry and wanted to confirm my understanding of how this should work. The documentation states that if you can pull images on the instance running Airbyte then it in turn should be able to pull images through the Interface when adding a custom connector.

In my case, I am running Airbyte (0.62.3) in Docker Compose. Airbyte fails to pull images from the private registry:

airbyte-worker                    | 2024-06-20 15:07:00 platform > us-docker.pkg.dev/gov34sdu-artifacts/gov/airbyte-source-netwatch:latest not found locally. Attempting to pull the image...
airbyte-worker                    | 2024-06-20 15:07:00 platform > Image does not exist.
airbyte-worker                    | Caused by: io.airbyte.workers.exception.WorkerException: Error while getting spec from image us-docker.pkg.dev/gov34sdu-artifacts/gov/airbyte-source-netwatch:latest
Caused by: io.airbyte.workers.exception.WorkerException: Could not find image: us-docker.pkg.dev/gov34sdu-artifacts/gov/airbyte-source-netwatch:latest
airbyte-worker                    | Caused by: io.airbyte.workers.exception.WorkerException: Could not find image: us-docker.pkg.dev/gov34sdu-artifacts/gov/airbyte-source-netwatch:latest```
Running the pull command on the instance, using the same user that runs Docker results in a successful image pull. After the image is pulled manually Airbyte is able to find and add it. Is this the intention or Airbyte should be able to pull the image from the registry on its own?

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1718899157514449) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["google-artifact-registry", "image-pulling", "private-registry", "docker-compose", "custom-connector"]
</sub>
2

I managed to implement a workaround, but I’m unsure whether I’m addressing an issue that shouldn’t exist in the first place or if the documentation simply lacks detailed instructions for setting up private registries.

The issue is with the airbyte-worker container and it not being configured to access private registries. To resolve this I had to:

• Extend the Airbyte worker image
◦ Include the binary docker-credential-gcr
◦ As airbyte user, run docker-credential-gcr configure-docker --registries="...."
• Docker Compose
◦ Mount Google’s Default Application Credentials json file on the worker
◦ Set the environment GOOGLE_APPLICATION_CREDENTIALS to point to the mount location on the worker
After making these changes I was able to add custom images through the Airbyte interface. I am not sure if this is the correct fix, but it did solve the issue for me.

The https://docs.airbyte.com/operator-guides/using-custom-connectors/#on-gcp---artifact-registry|documentation doesn’t really instructs you to that. It assumes credentials are passed implicitly to the containers, but this is not the case - at least in my scenario.