2023-01-31 23:02:03 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed. 
errors: $.client_secret: object found, string expected, $.refresh_token: object found, string expected
2023-01-31 23:02:04 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):138 - Cloud storage job log path: /workspace/467536/0/logs.log
2023-01-31 23:02:03 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed. 
errors: $.api_key: is missing but it is required, $.credentials_title: must be a constant value API Key Credentials
2023-01-31 23:02:03 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed. 
errors: $.access_token: is missing but it is required, $.credentials_title: must be a constant value Private App Credentials
2023-01-31 23:02:03 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed. 
errors: $.format_type: does not have a value in the enumeration [Avro], $.compression_codec: is missing but it is required
2023-01-31 23:02:03 [32mINFO[m i.a.v.j.JsonSchemaValidator(test):130 - JSON schema validation failed. 
errors: $.flattening: is missing but it is required, $.format_type: does not have a value in the enumeration [CSV]
2023-01-31 23:02:04 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.40.28
2023-01-31 23:02:04 [32mINFO[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to save workflow id for cancellation
2023-01-31 23:02:04 [32mINFO[m i.a.w.s.LauncherWorker(lambda$run$3):180 - Creating orchestrator-repl-job-467536-attempt-0 for attempt number: 0
2023-01-31 23:02:04 [32mINFO[m i.a.w.s.LauncherWorker(killRunningPodsForConnection):268 - Successfully deleted all running pods for the connection!
2023-01-31 23:02:04 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(create):440 - Waiting for pod to be running...
2023-01-31 23:03:55 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(create):463 - Pod default/orchestrator-repl-job-467536-attempt-0 is running on 10.0.105.106
2023-01-31 23:03:55 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):483 - Uploading file: envMap.json
2023-01-31 23:03:55 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):491 - kubectl cp /tmp/ed79070c-71e4-411e-a491-8f9dc3ad4554/envMap.json default/orchestrator-repl-job-467536-attempt-0:/config/envMap.json -c init
2023-01-31 23:03:55 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):494 - Waiting for kubectl cp to complete
2023-01-31 23:03:58 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):501 - kubectl cp complete, closing process
2023-01-31 23:03:58 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):483 - Uploading file: jobRunConfig.json
2023-01-31 23:03:58 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):491 - kubectl cp /tmp/ded25d1f-8e53-4a65-9aa1-451093bc6ad3/jobRunConfig.json default/orchestrator-repl-job-467536-attempt-0:/config/jobRunConfig.json -c init
2023-01-31 23:03:58 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):494 - Waiting for kubectl cp to complete
2023-01-31 23:04:05 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):501 - kubectl cp complete, closing process
2023-01-31 23:04:05 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):483 - Uploading file: application.txt
2023-01-31 23:04:05 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):491 - kubectl cp /tmp/a4c329ca-8853-4dec-9b4a-55847064ad08/application.txt default/orchestrator-repl-job-467536-attempt-0:/config/application.txt -c init
2023-01-31 23:04:05 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):494 - Waiting for kubectl cp to complete
2023-01-31 23:04:09 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):501 - kubectl cp complete, closing process
2023-01-31 23:04:09 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):483 - Uploading file: destinationLauncherConfig.json
2023-01-31 23:04:09 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):491 - kubectl cp /tmp/c8bb8404-7d9c-4c3d-9b00-a5e88db7799c/destinationLauncherConfig.json default/orchestrator-repl-job-467536-attempt-0:/config/destinationLauncherConfig.json -c init
2023-01-31 23:04:09 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):494 - Waiting for kubectl cp to complete
2023-01-31 23:04:11 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):501 - kubectl cp complete, closing process
2023-01-31 23:04:11 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):483 - Uploading file: sourceLauncherConfig.json
2023-01-31 23:04:11 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):491 - kubectl cp /tmp/d1d2dfad-7bb5-4705-a313-f9882d531ce4/sourceLauncherConfig.json default/orchestrator-repl-job-467536-attempt-0:/config/sourceLauncherConfig.json -c init
2023-01-31 23:04:11 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):494 - Waiting for kubectl cp to complete
2023-01-31 23:04:15 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):501 - kubectl cp complete, closing process
2023-01-31 23:04:15 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):483 - Uploading file: input.json
2023-01-31 23:04:15 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):491 - kubectl cp /tmp/89ae28d8-341b-483a-9186-709bc80ed52e/input.json default/orchestrator-repl-job-467536-attempt-0:/config/input.json -c init
2023-01-31 23:04:15 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):494 - Waiting for kubectl cp to complete
2023-01-31 23:04:15 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):501 - kubectl cp complete, closing process
2023-01-31 23:04:15 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):483 - Uploading file: KUBE_POD_INFO
2023-01-31 23:04:15 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):491 - kubectl cp /tmp/17280b10-43ed-4f55-954f-7f77c939acbd/KUBE_POD_INFO default/orchestrator-repl-job-467536-attempt-0:/config/KUBE_POD_INFO -c init
2023-01-31 23:04:15 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):494 - Waiting for kubectl cp to complete
2023-01-31 23:04:21 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):501 - kubectl cp complete, closing process
2023-01-31 23:04:21 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):483 - Uploading file: FINISHED_UPLOADING
2023-01-31 23:04:21 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):491 - kubectl cp /tmp/7f1b8218-d673-4b74-96e2-8ede3bc6f2af/FINISHED_UPLOADING default/orchestrator-repl-job-467536-attempt-0:/config/FINISHED_UPLOADING -c init
2023-01-31 23:04:21 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):494 - Waiting for kubectl cp to complete
2023-01-31 23:04:26 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(copyFilesToKubeConfigVolumeMain):501 - kubectl cp complete, closing process
2023-02-01 01:42:24 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$6):231 - Running sync worker cancellation...
2023-02-01 01:42:24 [33mWARN[m i.a.w.s.LauncherWorker(killRunningPodsForConnection):253 - There are currently running pods for the connection: [orchestrator-repl-job-467536-attempt-0]. Killing these pods to enforce one execution at a time.
2023-02-01 01:42:24 [32mINFO[m i.a.w.s.LauncherWorker(killRunningPodsForConnection):256 - Attempting to delete pods: [orchestrator-repl-job-467536-attempt-0]
2023-02-01 01:42:24 [32mINFO[m i.a.w.s.LauncherWorker(killRunningPodsForConnection):261 - Waiting for deletion...
2023-02-01 01:42:24 [32mINFO[m i.a.w.p.AsyncOrchestratorPodProcess(computeExitValue):129 - State Store missing status. Orchestrator pod orchestrator-repl-job-467536-attempt-0 non-existent. Assume failure.
2023-02-01 01:42:24 [32mINFO[m i.a.w.s.LauncherWorker(lambda$run$3):227 - Destroying process due to cancellation.
2023-02-01 01:42:24 [33mWARN[m i.a.w.p.AsyncOrchestratorPodProcess(destroy):186 - Wasn't able to delete pod orchestrator-repl-job-467536-attempt-0 from namespace default
2023-02-01 01:42:24 [32mINFO[m i.a.c.t.TemporalUtils(withBackgroundHeartbeat):316 - Stopping temporal heartbeating...
2023-02-01 01:42:24 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$5):198 - Completing future exceptionally...
java.lang.RuntimeException: io.airbyte.workers.exception.WorkerException: Launcher replication-orchestrator was cancelled.
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:314) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	at io.airbyte.workers.sync.LauncherWorker.run(LauncherWorker.java:116) ~[io.airbyte-airbyte-commons-worker-0.40.28.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: io.airbyte.workers.exception.WorkerException: Launcher replication-orchestrator was cancelled.
	at io.airbyte.workers.sync.LauncherWorker.lambda$run$3(LauncherWorker.java:232) ~[io.airbyte-airbyte-commons-worker-0.40.28.jar:?]
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:309) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	... 3 more
Caused by: java.util.concurrent.CancellationException
	at io.airbyte.workers.sync.LauncherWorker.lambda$run$3(LauncherWorker.java:207) ~[io.airbyte-airbyte-commons-worker-0.40.28.jar:?]
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:309) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	... 3 more
2023-02-01 01:42:24 [32mINFO[m i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...
2023-02-01 01:42:24 [32mINFO[m i.a.c.t.TemporalUtils(withBackgroundHeartbeat):283 - Stopping temporal heartbeating...
2023-02-01 01:42:24 [33mWARN[m i.t.i.a.ActivityTaskExecutors$BaseActivityTaskExecutor(execute):114 - Activity failure. ActivityId=7e47430f-eb35-30ac-b644-88119c09cbad, activityType=Replicate, attempt=1
java.lang.RuntimeException: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: java.lang.RuntimeException: io.airbyte.workers.exception.WorkerException: Launcher replication-orchestrator was cancelled.
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:281) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	at io.airbyte.workers.temporal.sync.ReplicationActivityImpl.replicate(ReplicationActivityImpl.java:158) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.activity.RootActivityInboundCallsInterceptor$POJOActivityInboundCallsInterceptor.executeActivity(RootActivityInboundCallsInterceptor.java:64) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.RootActivityInboundCallsInterceptor.execute(RootActivityInboundCallsInterceptor.java:43) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.ActivityTaskExecutors$BaseActivityTaskExecutor.execute(ActivityTaskExecutors.java:95) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.activity.ActivityTaskHandlerImpl.handle(ActivityTaskHandlerImpl.java:92) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handleActivity(ActivityWorker.java:241) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:206) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:179) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.17.0.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: java.lang.RuntimeException: io.airbyte.workers.exception.WorkerException: Launcher replication-orchestrator was cancelled.
	at io.temporal.serviceclient.CheckedExceptionWrapper.wrap(CheckedExceptionWrapper.java:57) ~[temporal-serviceclient-1.17.0.jar:?]
	at io.temporal.internal.sync.WorkflowInternal.wrap(WorkflowInternal.java:461) ~[temporal-sdk-1.17.0.jar:?]
	at io.temporal.activity.Activity.wrap(Activity.java:52) ~[temporal-sdk-1.17.0.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:167) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	at io.airbyte.workers.temporal.sync.ReplicationActivityImpl.lambda$replicate$3(ReplicationActivityImpl.java:203) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:276) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	... 14 more
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: io.airbyte.workers.exception.WorkerException: Launcher replication-orchestrator was cancelled.
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:161) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	at io.airbyte.workers.temporal.sync.ReplicationActivityImpl.lambda$replicate$3(ReplicationActivityImpl.java:203) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:276) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	... 14 more
Caused by: java.lang.RuntimeException: io.airbyte.workers.exception.WorkerException: Launcher replication-orchestrator was cancelled.
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:314) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	at io.airbyte.workers.sync.LauncherWorker.run(LauncherWorker.java:116) ~[io.airbyte-airbyte-commons-worker-0.40.28.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	... 1 more
Caused by: io.airbyte.workers.exception.WorkerException: Launcher replication-orchestrator was cancelled.
	at io.airbyte.workers.sync.LauncherWorker.lambda$run$3(LauncherWorker.java:232) ~[io.airbyte-airbyte-commons-worker-0.40.28.jar:?]
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:309) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	at io.airbyte.workers.sync.LauncherWorker.run(LauncherWorker.java:116) ~[io.airbyte-airbyte-commons-worker-0.40.28.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	... 1 more
Caused by: java.util.concurrent.CancellationException
	at io.airbyte.workers.sync.LauncherWorker.lambda$run$3(LauncherWorker.java:207) ~[io.airbyte-airbyte-commons-worker-0.40.28.jar:?]
	at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:309) ~[io.airbyte-airbyte-commons-temporal-0.40.28.jar:?]
	at io.airbyte.workers.sync.LauncherWorker.run(LauncherWorker.java:116) ~[io.airbyte-airbyte-commons-worker-0.40.28.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$5(TemporalAttemptExecution.java:195) ~[io.airbyte-airbyte-workers-0.40.28.jar:?]
	... 1 more
2023-02-01 01:42:25 [32mINFO[m i.a.w.s.LauncherWorker(killRunningPodsForConnection):268 - Successfully deleted all running pods for the connection!
2023-02-01 01:42:25 [32mINFO[m i.a.w.s.LauncherWorker(cancel):304 - Successfully cancelled process.
2023-02-01 01:42:25 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$6):235 - Interrupting worker thread...
2023-02-01 01:42:25 [32mINFO[m i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$6):238 - Cancelling completable future...
2023-02-01 01:42:25 [33mWARN[m i.a.c.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled.
2023-02-01 01:42:29 [32mINFO[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to get state
2023-02-01 01:42:29 [32mINFO[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to create or update state
2023-02-01 01:42:36 [32mINFO[m i.a.a.c.AirbyteApiClient(retryWithJitter):172 - Attempt 0 to get job info light
2023-02-01 01:42:36 [32mINFO[m i.a.w.t.s.ReplicationActivityImpl(getContainerLauncherWorkerFactory):333 - received response from from jobsApi.getJobInfoLight: class JobInfoLightRead {
    job: class JobRead {
        id: 467997
        configType: sync
        configId: 9e1882da-092a-47df-938d-7b6c5735e8b7
        createdAt: 1675215755
        updatedAt: 1675215755
        status: running
        resetConfig: null
    }
}