Error creating or updating state in Airbyte platform with Postgresql connector

Summary

The error message ‘createOrUpdateState call failed with: 500 - Internal Server Error: null’ indicates a NullPointerException in the Airbyte platform while replicating data from Postgresql to DuckDB. The stacktrace shows the issue originating from the StateApi and SyncPersistenceImpl classes.


Question

Hi, error conn Airbyte (Postgresql) -> DucKdb(MotherDuckDB) = createOrUpdateState call failed with: 500 - {“message”:“Internal Server Error: null”,“exceptionClassName”:“java.lang.NullPointerException”,“exceptionStack”:[“java.lang.NullPointerException”]} 520 2024-04-10 22:38:55 platform > failures: [ {
545
“failureOrigin” : “replication”,
546
“internalMessage” : “createOrUpdateState call failed with: 500 - {"message":"Internal Server Error: null","exceptionClassName":"java.lang.NullPointerException","exceptionStack":["java.lang.NullPointerException"]}”,
547
“externalMessage” : “Something went wrong during replication”,
548
“metadata” : {
549
“attemptNumber” : 3,
550
“jobId” : 80460
551
},
552
“stacktrace” : “io.airbyte.api.client.invoker.generated.ApiException: createOrUpdateState call failed with: 500 - {"message":"Internal Server Error: null","exceptionClassName":"java.lang.NullPointerException","exceptionStack":["java.lang.NullPointerException"]}\n\tat io.airbyte.api.client.generated.StateApi.getApiException(StateApi.java:72)\n\tat io.airbyte.api.client.generated.StateApi.createOrUpdateStateWithHttpInfo(StateApi.java:112)\n\tat io.airbyte.api.client.generated.StateApi.createOrUpdateState(StateApi.java:90)\n\tat io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.doFlushState(SyncPersistence.kt:322)\n\tat io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.access$doFlushState(SyncPersistence.kt:67)\n\tat io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl$close$3.invoke(SyncPersistence.kt:225)\n\tat io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl$close$3.invoke(SyncPersistence.kt:224)\n\tat io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.retryWithJitterThrows$lambda$7(SyncPersistence.kt:388)\n\tat io.airbyte.api.client.AirbyteApiClient.retryWithJitterThrows(AirbyteApiClient.java:298)\n\tat io.airbyte.api.client.AirbyteApiClient.retryWithJitterThrows(AirbyteApiClient.java:261)\n\tat io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.retryWithJitterThrows(SyncPersistence.kt:388)\n\tat io.airbyte.workers.internal.syncpersistence.SyncPersistenceImpl.close(SyncPersistence.kt:224)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.run(BufferedReplicationWorker.java:169)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.run(BufferedReplicationWorker.java:56)\n\tat io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:142)\n\tat io.airbyte.workers.temporal.sync.ReplicationActivityImpl.lambda$replicateV2$2(ReplicationActivityImpl.java:190)\n\tat io.airbyte.commons.temporal.HeartbeatUtils.withBackgroundHeartbeat(HeartbeatUtils.java:57)\n\tat io.airbyte.workers.temporal.sync.ReplicationActivityImpl.replicateV2(ReplicationActivityImpl.java:158)\n\tat java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)\n\tat java.base/java.lang.reflect.Method.invoke(Method.java:580)\n\tat io.temporal.internal.activity.RootActivityInboundCallsInterceptor$POJOActivityInboundCallsInterceptor.executeActivity(RootActivityInboundCallsInterceptor.java:64)\n\tat io.temporal.internal.activity.RootActivityInboundCallsInterceptor.execute(RootActivityInboundCallsInterceptor.java:43)\n\tat io.temporal.internal.activity.ActivityTaskExecutors$BaseActivityTaskExecutor.execute(ActivityTaskExecutors.java:107)\n\tat io.temporal.internal.activity.ActivityTaskHandlerImpl.handle(ActivityTaskHandlerImpl.java:124)\n\tat io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handleActivity(ActivityWorker.java:278)\n\tat io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:243)\n\tat io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:216)\n\tat io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:105)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n”,
553
“timestamp” : 1712788735721
554
} ]
555
2024-04-10 22:38:55 platform >
556
2024-04-10 22:38:55 platform > ----- END REPLICATION -----
557
2024-04-10 22:38:55 platform >
558
2024-04-10 22:38:56 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=10, successivePartialFailureLimit=1000, totalPartialFailureLimit=10, successiveCompleteFailures=4, totalCompleteFailures=4, successivePartialFailures=0, totalPartialFailures=0)
559
Backoff before next attempt: 4 minutes 30 seconds



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["error", "create-or-update-state", "airbyte-platform", "postgresql-connector", "duckdb"]

succeeded in extracting the 10 rows but failed to load, did it work for anyone before?

Thank you for your comments