Unable to get image error

vivek · July 11, 2022, 8:21pm

Is this your first time deploying Airbyte?: Yes
OS Version / Instance: Windows
Memory / Disk: 32 GB / 500 GB
Deployment: Docker
Airbyte Version: Latest 0.39
Source name/version:
Destination name/version:
Step: The issue is happening during sync, creating the connection or a new source?
Description: Remove this with the description of your problem.

I’m new to airbyte. Cloned the repo and did docker compose up. The UI comes up and everything looks good.

However, I am unable to setup any connection. Any source I pick, the error I see is:

Did not find image locally
Unable to get image

This happens even when I pull the image locally via docker pull. I tried exchange rates and postgres source, also created a custom source but to no vain

Any pointers on what I’m missing please?

Regards,
Vivek

natalyjazzviolin · July 11, 2022, 8:52pm

Hi @Vivek, welcome to the Airbyte community!

I understand you’re seeing these errors on the frontend, right? Could you please give me the error logs the server is putting out? Are you running Airbyte in dev mode?

vivek · July 13, 2022, 6:31pm

Here are the logs from the UI:

	2022-07-13 18:25:49 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/a659bbea-455e-43cb-ae88-296eb8c5ecb8/0/logs.log
2022-07-13 18:25:49 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.39.35-alpha
2022-07-13 18:25:50 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):43 - Using default value for environment variable USE_STREAM_CAPABLE_STATE: 'false'
2022-07-13 18:25:50 INFO i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/source-postgres:0.4.30 exists...
2022-07-13 18:25:50 INFO i.a.c.i.LineGobbler(voidCall):82 - airbyte/source-postgres:0.4.30 not found locally. Attempting to pull the image...
2022-07-13 18:25:50 INFO i.a.c.i.LineGobbler(voidCall):82 - Image does not exist.
2022-07-13 18:25:50 ERROR i.a.w.g.DefaultCheckConnectionWorker(run):93 - Unexpected error while checking connection: 
io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-postgres:0.4.30
	at io.airbyte.workers.process.DockerProcessFactory.create(DockerProcessFactory.java:97) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
	at io.airbyte.workers.process.AirbyteIntegrationLauncher.check(AirbyteIntegrationLauncher.java:84) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
	at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:57) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
	at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
2022-07-13 18:25:50 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):161 - Completing future exceptionally...
io.airbyte.workers.exception.WorkerException: Unexpected error while getting checking connection.
	at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:94) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
	at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-postgres:0.4.30

logs from docker compose up prompt ( I removed -d flag) :

airbyte-worker      | 2022-07-13 18:25:49 INFO i.a.c.EnvConfigs(getEnvOrDefault):967 - Using default value for environment variable CONTAINER_ORCHESTRATOR_ENABLED: 'false'
airbyte-worker      | 2022-07-13 18:25:49 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/a659bbea-455e-43cb-ae88-296eb8c5ecb8/0/logs.log
airbyte-worker      | 2022-07-13 18:25:49 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.39.35-alpha
airbyte-worker      | 2022-07-13 18:25:50 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):43 - Using default value for environment variable USE_STREAM_CAPABLE_STATE: 'false'
airbyte-worker      | 2022-07-13 18:25:50 INFO i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/source-postgres:0.4.30 exists...
airbyte-worker      | 2022-07-13 18:25:50 INFO i.a.c.i.LineGobbler(voidCall):82 - airbyte/source-postgres:0.4.30 not found locally. Attempting to pull the image...
airbyte-worker      | 2022-07-13 18:25:50 INFO i.a.c.i.LineGobbler(voidCall):82 - Image does not exist.
airbyte-worker      | 2022-07-13 18:25:50 ERROR i.a.w.g.DefaultCheckConnectionWorker(run):93 - Unexpected error while checking connection:
airbyte-worker      | io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-postgres:0.4.30
airbyte-worker      |   at io.airbyte.workers.process.DockerProcessFactory.create(DockerProcessFactory.java:97) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.process.AirbyteIntegrationLauncher.check(AirbyteIntegrationLauncher.java:84) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:57) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at java.lang.Thread.run(Thread.java:833) [?:?]
airbyte-worker      | 2022-07-13 18:25:50 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):161 - Completing future exceptionally...
airbyte-worker      | io.airbyte.workers.exception.WorkerException: Unexpected error while getting checking connection.
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:94) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at java.lang.Thread.run(Thread.java:833) [?:?]
airbyte-worker      | Caused by: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-postgres:0.4.30
airbyte-worker      |   at io.airbyte.workers.process.DockerProcessFactory.create(DockerProcessFactory.java:97) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.process.AirbyteIntegrationLauncher.check(AirbyteIntegrationLauncher.java:84) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:57) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   ... 3 more
airbyte-worker      | 2022-07-13 18:25:50 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...
airbyte-worker      | 2022-07-13 18:25:50 WARN i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=451f4991-e798-3c71-9d71-3994958d7dfe, activityType=Run, attempt=1
airbyte-worker      | java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Unexpected error while getting checking connection.
airbyte-worker      |   at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
airbyte-worker      |   at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
airbyte-worker      |   at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.temporal.check.connection.CheckConnectionActivityImpl.run(CheckConnectionActivityImpl.java:76) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
airbyte-worker      |   at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
airbyte-worker      |   at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
airbyte-worker      |   at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
airbyte-worker      |   at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
airbyte-worker      |   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
airbyte-worker      |   at java.lang.Thread.run(Thread.java:833) [?:?]
airbyte-worker      | Caused by: io.airbyte.workers.exception.WorkerException: Unexpected error while getting checking connection.
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:94) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   ... 1 more
airbyte-worker      | Caused by: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-postgres:0.4.30
airbyte-worker      |   at io.airbyte.workers.process.DockerProcessFactory.create(DockerProcessFactory.java:97) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.process.AirbyteIntegrationLauncher.check(AirbyteIntegrationLauncher.java:84) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:57) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   ... 1 more
airbyte-worker      | 2022-07-13 18:25:50 ERROR i.t.i.s.WorkflowExecuteRunnable(logWorkflowExecutionException):125 - Workflow execution failure WorkflowId=6ebde5cd-4fb8-4edd-879e-41073d5f55ec, RunId=49d05238-eb81-4473-bef1-d9eb7b702b3d, WorkflowType=CheckConnectionWorkflow
airbyte-worker      | io.temporal.failure.ActivityFailure: scheduledEventId=5, startedEventId=6, activityType='Run', activityId='451f4991-e798-3c71-9d71-3994958d7dfe', identity='1@4213990ce144', retryState=RETRY_STATE_MAXIMUM_ATTEMPTS_REACHED
airbyte-worker      |   at java.lang.Thread.getStackTrace(Thread.java:1610) ~[?:?]
airbyte-worker      |   at io.temporal.internal.sync.ActivityStubBase.execute(ActivityStubBase.java:48) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.sync.ActivityInvocationHandler.lambda$getActivityFunc$0(ActivityInvocationHandler.java:77) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.sync.ActivityInvocationHandlerBase.invoke(ActivityInvocationHandlerBase.java:70) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at jdk.proxy2.$Proxy32.run(Unknown Source) ~[?:?]
airbyte-worker      |   at io.airbyte.workers.temporal.check.connection.CheckConnectionWorkflowImpl.run(CheckConnectionWorkflowImpl.java:25) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
airbyte-temporal    | {"level":"info","ts":"2022-07-13T18:25:50.759Z","msg":"Get dynamic config","name":"frontend.sendRawWorkflowHistory","value":false,"default-value":false,"logging-call-at":"config.go:79"}
airbyte-worker      |   at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
airbyte-worker      |   at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
airbyte-worker      |   at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
airbyte-worker      |   at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation$RootWorkflowInboundCallsInterceptor.execute(POJOWorkflowImplementationFactory.java:317) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation.execute(POJOWorkflowImplementationFactory.java:292) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.sync.WorkflowExecuteRunnable.run(WorkflowExecuteRunnable.java:72) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.sync.SyncWorkflow.lambda$start$0(SyncWorkflow.java:137) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.sync.CancellationScopeImpl.run(CancellationScopeImpl.java:101) [temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.sync.WorkflowThreadImpl$RunnableWrapper.run(WorkflowThreadImpl.java:111) [temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
airbyte-worker      |   at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
airbyte-worker      |   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
airbyte-worker      |   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
airbyte-worker      |   at java.lang.Thread.run(Thread.java:833) [?:?]
airbyte-worker      | Caused by: io.temporal.failure.ApplicationFailure: message='io.airbyte.workers.exception.WorkerException: Unexpected error while getting checking connection.', type='java.util.concurrent.ExecutionException', nonRetryable=false
airbyte-worker      |   at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
airbyte-worker      |   at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
airbyte-worker      |   at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.temporal.check.connection.CheckConnectionActivityImpl.run(CheckConnectionActivityImpl.java:76) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method:0) ~[?:?]
airbyte-worker      |   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
airbyte-worker      |   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
airbyte-worker      |   at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
airbyte-worker      |   at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
airbyte-worker      |   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
airbyte-worker      |   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
airbyte-worker      |   at java.base/java.lang.Thread.run(Thread.java:833) ~[?:?]
airbyte-worker      | Caused by: io.temporal.failure.ApplicationFailure: message='Unexpected error while getting checking connection.', type='io.airbyte.workers.exception.WorkerException', nonRetryable=false
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:94) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-server      | 2022-07-13 18:25:50 INFO i.a.s.RequestLogger(filter):112 - REQ 172.22.0.5 POST 200 /api/v1/scheduler/sources/check_connection - {"connectionConfiguration":"REDACTED","sourceDefinitionId":"decd338e-5647-4c0b-adf4-da0e75f5a750"}
airbyte-worker      |   at java.base/java.lang.Thread.run(Thread.java:833) ~[?:?]
airbyte-worker      | Caused by: io.temporal.failure.ApplicationFailure: message='Could not find image: airbyte/source-postgres:0.4.30', type='io.airbyte.workers.exception.WorkerException', nonRetryable=false
airbyte-webapp      | 172.22.0.1 - - [13/Jul/2022:18:25:50 +0000] "POST /api/v1/scheduler/sources/check_connection HTTP/1.1" 200 7525 "http://localhost:8000/workspaces/659e73ff-e0a1-42db-b8e3-4da171e3dca7/connections/new-connection" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" "-"
airbyte-worker      |   at io.airbyte.workers.process.DockerProcessFactory.create(DockerProcessFactory.java:97) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.process.AirbyteIntegrationLauncher.check(AirbyteIntegrationLauncher.java:84) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:57) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.35-alpha.jar:?]
airbyte-worker      |   at java.base/java.lang.Thread.run(Thread.java:833) ~[?:?]

Steps I have performed so far:

git clone to directory
docker compose up
try to set up a connection with local postgres instance

vivek · July 13, 2022, 6:42pm

I have also tried docker compose down -v and then docker compose up

fauh45 · July 14, 2022, 5:05am

Done the same thing and I do get the same error as well, here are my logs

airbyte-webapp      | 172.20.0.1 - - [14/Jul/2022:05:00:28 +0000] "GET /api/v1/health HTTP/1.1" 200 18 "http://localhost:8000/workspaces/744bea5b-131f-4931-ab72-b989d57c18e9/connections/new-connection" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.114 Safari/537.36" "-"
airbyte-worker      | 2022-07-14 05:00:28 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):43 - Using default value for environment variable USE_STREAM_CAPABLE_STATE: 'false'
airbyte-worker      | 2022-07-14 05:00:28 INFO i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/source-postgres:0.4.31 exists...
airbyte-worker      | 2022-07-14 05:00:28 INFO i.a.c.i.LineGobbler(voidCall):82 - airbyte/source-postgres:0.4.31 not found locally. Attempting to pull the image...
airbyte-worker      | 2022-07-14 05:00:28 INFO i.a.c.i.LineGobbler(voidCall):82 - Image does not exist.
airbyte-worker      | 2022-07-14 05:00:28 ERROR i.a.w.g.DefaultCheckConnectionWorker(run):93 - Unexpected error while checking connection: 
airbyte-worker      | io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-postgres:0.4.31
airbyte-worker      | 	at io.airbyte.workers.process.DockerProcessFactory.create(DockerProcessFactory.java:97) ~[io.airbyte-airbyte-workers-0.39.36-alpha.jar:?]
airbyte-worker      | 	at io.airbyte.workers.process.AirbyteIntegrationLauncher.check(AirbyteIntegrationLauncher.java:84) ~[io.airbyte-airbyte-workers-0.39.36-alpha.jar:?]
airbyte-worker      | 	at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:57) ~[io.airbyte-airbyte-workers-0.39.36-alpha.jar:?]
airbyte-worker      | 	at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.36-alpha.jar:?]
airbyte-worker      | 	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.36-alpha.jar:?]
airbyte-worker      | 	at java.lang.Thread.run(Thread.java:833) [?:?]
airbyte-worker      | 2022-07-14 05:00:28 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):161 - Completing future exceptionally...
airbyte-worker      | io.airbyte.workers.exception.WorkerException: Unexpected error while getting checking connection.
airbyte-worker      | 	at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:94) ~[io.airbyte-airbyte-workers-0.39.36-alpha.jar:?]
airbyte-worker      | 	at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:31) ~[io.airbyte-airbyte-workers-0.39.36-alpha.jar:?]
airbyte-worker      | 	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.36-alpha.jar:?]
airbyte-worker      | 	at java.lang.Thread.run(Thread.java:833) [?:?]
airbyte-worker      | Caused by: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-postgres:0.4.31

natalyjazzviolin · July 14, 2022, 2:29pm

@vivek, this looks like a Windows/Docker configuration issue. Are you using Docker Desktop or WSL? Is there enough disk space for the image?

@fauh45, are you also on WIndows?

vivek · July 14, 2022, 4:40pm

I am using Docker Desktop with WSL2 for docker enabled.

There is sufficient space for the image. I also pulled the image separately:

fauh45 · July 15, 2022, 2:03am

I use Ubuntu 20.04, with rootless Docker, and there are also plenty for the image to pull. I even tried it manually as well.

fauh45 · July 15, 2022, 2:11am

Ah a little update on the error. I tried to running it with root and it works as intended.

There must be a little problem running it in rootless mode.

vivek · July 15, 2022, 3:49am

I tried running Docker desktop and the docker compose step with Admin privileges but still no luck I still get could not find image even though the image has been downloaded manually

vivek · July 15, 2022, 3:54am

Does it expect the image to be present inside the container ?! that would invalidate my understanding

natalyjazzviolin · July 15, 2022, 1:51pm

@fauh45 so glad to hear it worked out for you!! I’ll file this away for future reference.

@vivek, could you try a complete reinstall of Docker Desktop? I found this worked for another Docker user, though it’s not the clearest comment:
https://forums.docker.com/t/docker-is-unable-to-find-image-locally/99504

vivek · July 15, 2022, 5:57pm

that’s the thing, docker IS able to find other images and download them successfully when I run the commands:

It’s only airbyte that’s not able to get docker to download/get images

natalyjazzviolin · July 19, 2022, 7:53pm

Thank you for the information @vivek, I am sorry this is taking so long to figure out. I’ve asked my team for insight on this and hope to hear back soon.

vivek · July 19, 2022, 7:53pm

Any help on this guys please ?

natalyjazzviolin · July 21, 2022, 10:16am

@vivek, could you please open an issue on GitHub? It looks like bugs like this are really common with WSL2. If you have access to a Linux machine, you might find it easier to get up and running there!

seyal84 · November 9, 2022, 10:53pm

Any update on this if it has been resolved ?

I am getting the same issue via docker running in Rhel , I am able to download images manually but unable to download via air yet.

hugozap · December 22, 2022, 6:18pm

I setup Airbyte on a remote virtual server running Fedora. I’m also getting the image not found error when trying to setup a Source. (Trying to pull the image manually with docker succeeds and I have enough disk space). Not sure how to troubleshoot this error.

 Checking if airbyte/source-file:0.2.32 exists...
airbyte-worker                    | 2022-12-22 18:09:33 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/source-file:0.2.32 not found locally. Attempting to pull the image...
airbyte-worker                    | 2022-12-22 18:09:34 INFO i.a.c.i.LineGobbler(voidCall):114 - Image does not exist.

ce-file:0.2.32
airbyte-worker                    |     at io.airbyte.workers.general.DefaultGetSpecWorker.run(DefaultGetSpecWorker.java:96) ~[io.airbyte-airbyte-commons-worker-0.40.26.jar:?]
airbyte-worker                    |     at io.airbyte.workers.general.DefaultGetSpecWorker.run(DefaultGetSpecWorker.java:37) ~[io.airbyte-airbyte-commons-worker-0.40.26.jar:?]
airbyte-worker                    |     at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.40.26.jar:?]
airbyte-worker                    |     at java.lang.Thread.run(Thread.java:1589) ~[?:?]
airbyte-worker                    | Caused by: io.airbyte.workers.exception.WorkerException: Could not find image: airbyte/source-file:0.2.32

lauripe · January 5, 2023, 5:30pm

I ran into the same with Docker on WSL2, and the issue was that the workers docker.sock mount didn’t match the hosts configuration.

Quick and dirty workaround was to change/override the key accordingly, e.g:

/mnt/wsl/shared-docker/docker.sock:/var/run/docker.sock

Topic		Replies	Views
Unknown error occurred while setting up connection 0.40.27 Connector Questions & Issues docker , connectors	2	405	January 12, 2023
Setting up a connection - parquet files locally Connector Questions & Issues connectors	1	174	March 3, 2023
Airbyte API Server Returning 404 for endpoint Connector Questions & Issues connectors	6	796	October 27, 2022
Error with api/v1/source_definitions/create when running on GCP Connector Questions & Issues connectors	6	493	July 14, 2022
Python source connectors Connector Questions & Issues connectors	1	493	August 15, 2022

Unable to get image error

Related Topics