Summary
The user is encountering an error related to a Workflow task not being scheduled yet in Temporal, causing issues with syncing connections. The error message indicates a FAILED_PRECONDITION due to the Workflow task not being scheduled.
Question
Has anyone encountered a error like Workflow task is scheduled yet, error from temporal? logs are below
2024-09-30 11:03:50 ERROR i.a.c.t.ConnectionManagerUtils(getWorkflowState):221 - Exception thrown while checking workflow state for connection id d7369ba4-984c-4efe-9691-8947e786cbd5
io.temporal.client.WorkflowQueryRejectedException: workflowId='connection_manager_d7369ba4-984c-4efe-9691-8947e786cbd5', runId='', workflowType='ConnectionManagerWorkflow'}
at io.temporal.client.WorkflowStubImpl.throwAsWorkflowFailureExceptionForQuery(WorkflowStubImpl.java:482) ~[temporal-sdk-1.22.3.jar:?]
at io.temporal.client.WorkflowStubImpl.query(WorkflowStubImpl.java:280) ~[temporal-sdk-1.22.3.jar:?]
at io.temporal.client.WorkflowInvocationHandler$SyncWorkflowInvocationHandler.queryWorkflow(WorkflowInvocationHandler.java:308) ~[temporal-sdk-1.22.3.jar:?]
at io.temporal.client.WorkflowInvocationHandler$SyncWorkflowInvocationHandler.invoke(WorkflowInvocationHandler.java:269) ~[temporal-sdk-1.22.3.jar:?]
at io.temporal.client.WorkflowInvocationHandler.invoke(WorkflowInvocationHandler.java:175) ~[temporal-sdk-1.22.3.jar:?]
at jdk.proxy2/jdk.proxy2.$Proxy95.getState(Unknown Source) ~[?:?]
at io.airbyte.commons.temporal.ConnectionManagerUtils.getWorkflowState(ConnectionManagerUtils.java:219) ~[io.airbyte-airbyte-commons-temporal-0.63.13.jar:?]
at io.airbyte.commons.temporal.ConnectionManagerUtils.isWorkflowStateRunning(ConnectionManagerUtils.java:227) ~[io.airbyte-airbyte-commons-temporal-0.63.13.jar:?]
at io.airbyte.commons.temporal.TemporalClient.startNewManualSync(TemporalClient.java:251) ~[io.airbyte-airbyte-commons-temporal-0.63.13.jar:?]
at io.airbyte.commons.server.scheduler.TemporalEventRunner.startNewManualSync(TemporalEventRunner.java:34) ~[io.airbyte-airbyte-commons-server-0.63.13.jar:?]
at io.airbyte.commons.server.handlers.SchedulerHandler.submitManualSyncToWorker(SchedulerHandler.java:845) ~[io.airbyte-airbyte-commons-server-0.63.13.jar:?]
at io.airbyte.commons.server.handlers.SchedulerHandler.syncConnection(SchedulerHandler.java:618) ~[io.airbyte-airbyte-commons-server-0.63.13.jar:?]
at io.airbyte.server.apis.ConnectionApiController.lambda$syncConnection$20(ConnectionApiController.java:285) ~[io.airbyte-airbyte-server-0.63.13.jar:?]
at io.airbyte.server.apis.ApiHelper.execute(ApiHelper.java:28) ~[io.airbyte-airbyte-server-0.63.13.jar:?]
at io.airbyte.server.apis.ConnectionApiController.syncConnection(ConnectionApiController.java:285) ~[io.airbyte-airbyte-server-0.63.13.jar:?]
at io.airbyte.server.apis.$ConnectionApiController$Definition$Exec.dispatch(Unknown Source) ~[io.airbyte-airbyte-server-0.63.13.jar:?]
at io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461) ~[micronaut-inject-4.5.4.jar:4.5.4]
at io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4326) ~[micronaut-inject-4.5.4.jar:4.5.4]
at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:271) ~[micronaut-router-4.5.4.jar:4.5.4]
at io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:39) ~[micronaut-router-4.5.4.jar:4.5.4]
at io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:490) ~[micronaut-http-server-4.5.4.jar:4.5.4]
at io.micronaut.http.server.RouteExecutor.lambda$callRoute$6(RouteExecutor.java:467) ~[micronaut-http-server-4.5.4.jar:4.5.4]
at io.micronaut.core.execution.ExecutionFlow.lambda$async$1(ExecutionFlow.java:87) ~[micronaut-core-4.5.4.jar:4.5.4]
at io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:211) ~[micronaut-core-4.5.4.jar:4.5.4]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Workflow task is not scheduled yet.
at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:268) ~[grpc-stub-1.64.0.jar:1.64.0]
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:249) ~[grpc-stub-1.64.0.jar:1.64.0]
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:167) ~[grpc-stub-1.64.0.jar:1.64.0]
at io.temporal.api.workflowservice.v1.WorkflowServiceGrpc$WorkflowServiceBlockingStub.queryWorkflow(WorkflowServiceGrpc.java:4259) ~[temporal-serviceclient-1.22.3.jar:?]
at io.temporal.internal.client.external.GenericWorkflowClientImpl.lambda$query$10(GenericWorkflowClientImpl.java:205) ~[temporal-sdk-1.22.3.jar:?]
at io.temporal.internal.retryer.GrpcSyncRetryer.retry(GrpcSyncRetryer.java:69) ~[temporal-serviceclient-1.22.3.jar:?]
at io.temporal.internal.retryer.GrpcRetryer.retryWithResult(GrpcRetryer.java:60) ~[temporal-serviceclient-1.22.3.jar:?]
at io.temporal.internal.client.external.GenericWorkflowClientImpl.query(GenericWorkflowClientImpl.java:200) ~[temporal-sdk-1.22.3.jar:?]
at io.temporal.internal.client.RootWorkflowClientInvoker.query(RootWorkflowClientInvoker.java:284) ~[temporal-sdk-1.22.3.jar:?]
at io.temporal.client.WorkflowStubImpl.query(WorkflowStubImpl.java:276) ~[temporal-sdk-1.22.3.jar:?]
Does anyone knows on how to resolve this? Sometimes everything is working fine, but then encountering issues while the syncing the connections.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.