Summary
The user is facing an error in the destination connector despite having a primary key in the source table. The error message indicates that the table type in the destination requires a primary key, leading to system errors and non-zero exit codes.
Question
Detination: Mariadb 10.5.21
Source: Mariadb 10.5.21
My sources table already has a primary key, but why i am getting the below error?
2024-11-22 07:06:18 replication-orchestrator > failures: [ {
“failureOrigin” : “destination”,
“failureType” : “system_error”,
“internalMessage” : “java.util.concurrent.CompletionException: java.lang.RuntimeException: java.sql.SQLSyntaxErrorException: This table type requires a primary key”,
“externalMessage” : “Something went wrong in the connector. See the logs for more details.”,
“metadata” : {
“attemptNumber” : 0,
“jobId” : 57,
“from_trace_message” : true,
“connector_command” : “write”
},
“stacktrace” : “java.util.concurrent.CompletionException: java.lang.RuntimeException: java.sql.SQLSyntaxErrorException: This table type requires a primary key\n\tat java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)\n\tat java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1770)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: java.lang.RuntimeException: java.sql.SQLSyntaxErrorException: This table type requires a primary key\n\tat io.airbyte.integrations.base.destination.typing_deduping.DefaultTyperDeduper.prepareTablesFuture$lambda$4(DefaultTyperDeduper.kt:264)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)\n\t… 3 more\nCaused by: java.sql.SQLSyntaxErrorException: This table type requires a primary key\n\tat com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)\n\tat com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)\n\tat com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)\n\tat com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:764)\n\tat com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:648)\n\tat com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:94)\n\tat com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java)\n\tat io.airbyte.cdk.db.jdbc.JdbcDatabase.executeWithinTransaction$lambda$1(JdbcDatabase.kt:46)\n\tat io.airbyte.cdk.db.jdbc.DefaultJdbcDatabase.execute(DefaultJdbcDatabase.kt:30)\n\tat io.airbyte.cdk.db.jdbc.JdbcDatabase.executeWithinTransaction(JdbcDatabase.kt:43)\n\tat io.airbyte.cdk.integrations.destination.jdbc.typing_deduping.JdbcDestinationHandler.execute(JdbcDestinationHandler.kt:180)\n\tat io.airbyte.integrations.base.destination.typing_deduping.DefaultTyperDeduper.prepareTablesFuture$lambda$4(DefaultTyperDeduper.kt:234)\n\t… 4 more\n”,
“timestamp” : 1732259101353
}, {
“failureOrigin” : “destination”,
“internalMessage” : “Destination process exited with non-zero exit code 1”,
“externalMessage” : “Something went wrong within the destination connector”,
“metadata” : {
“attemptNumber” : 0,
“jobId” : 57,
“connector_command” : “write”
},
“stacktrace” : “io.airbyte.workers.internal.exception.DestinationException: Destination process exited with non-zero exit code 1\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromDestination(BufferedReplicationWorker.java:502)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:215)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n”,
“timestamp” : 1732259101453
}, {
“failureOrigin” : “destination”,
“internalMessage” : “Destination process message delivery failed”,
“externalMessage” : “Something went wrong within the destination connector”,
“metadata” : {
“attemptNumber” : 0,
“jobId” : 57,
“connector_command” : “write”
},
“stacktrace” : “io.airbyte.workers.internal.exception.DestinationException: Destination process message delivery failed\n\tat io.airbyte.workers.general.BufferedReplicationWorker.writeToDestination(BufferedReplicationWorker.java:453)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithTimeout$5(BufferedReplicationWorker.java:243)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: java.lang.IllegalStateException: No exit code found.\n\tat io.airbyte.workers.internal.ContainerIOHandle.getExitCode(ContainerIOHandle.kt:104)\n\tat io.airbyte.workers.internal.LocalContainerAirbyteSource.getExitValue(LocalContainerAirbyteSource.kt:89)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.writeToDestination(BufferedReplicationWorker.java:442)\n\t… 5 more\n”,
“timestamp” : 1732259101816
}, {
“failureOrigin” : “source”,
“internalMessage” : “Source process read attempt failed”,
“externalMessage” : “Something went wrong within the source connector”,
“metadata” : {
“attemptNumber” : 0,
“jobId” : 57,
“connector_command” : “read”
},
“stacktrace” : “io.airbyte.workers.internal.exception.SourceException: Source process read attempt failed\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(BufferedReplicationWorker.java:375)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithHeartbeatCheck$3(BufferedReplicationWorker.java:222)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: java.lang.IllegalStateException: No exit code found.\n\tat io.airbyte.workers.internal.ContainerIOHandle.getExitCode(ContainerIOHandle.kt:104)\n\tat io.airbyte.workers.internal.LocalContainerAirbyteSource.getExitValue(LocalContainerAirbyteSource.kt:89)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(BufferedReplicationWorker.java:355)\n\t… 5 more\n”,
“timestamp” : 1732259117993
}, {
“failureOrigin” : “replication”,
“internalMessage” : “io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.”,
“externalMessage” : “Something went wrong during replication”,
“metadata” : {
“attemptNumber” : 0,
“jobId” : 57
},
“stacktrace” : “java.lang.RuntimeException: io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.\n\tat io.airbyte.workers.general.BufferedReplicationWorker$CloseableWithTimeout.lambda$close$0(BufferedReplicationWorker.java:547)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithTimeout$5(BufferedReplicationWorker.java:243)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.\n\tat io.airbyte.workers.internal.LocalContainerAirbyteDestination.close(LocalContainerAirbyteDestination.kt:60)\n\tat io.airbyte.workers.general.BufferedReplicationWorker$CloseableWithTimeout.lambda$close$0(BufferedReplicationWorker.java:545)\n\t… 5 more\n”,
“timestamp” : 1732259118066
} ]
2024-11-22 07:06:18 replication-orchestrator >
2024-11-22 07:06:18 replication-orchestrator > ----- END REPLICATION -----
2024-11-22 07:06:18 replication-orchestrator >
2024-11-22 07:06:18 replication-orchestrator > Returning output…
2024-11-22 07:06:18 platform > Closing Segment analytics client…
2024-11-22 07:06:18 platform > Waiting for Segment analytic client to flush enqueued messages…
2024-11-22 07:06:18 platform > Segment analytic client flush complete.
2024-11-22 07:06:18 platform > Segment analytics client closed. No new events will be accepted.
2024-11-22 07:06:40 platform > Workload 0faae765-63f0-4246-af5f-6cdc5a8c0fb0_57_0_sync has returned a terminal status of failure. Fetching output…
2024-11-22 07:06:40 platform > No post-replication operation(s) to perform.
2024-11-22 07:06:40 platform >
2024-11-22 07:06:40 platform > ----- START POST REPLICATION OPERATIONS -----
2024-11-22 07:06:40 platform >
2024-11-22 07:06:40 platform >
2024-11-22 07:06:40 platform > ----- END POST REPLICATION OPERATIONS -----
2024-11-22 07:06:40 platform >
2024-11-22 07:06:40 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=10, successivePartialFailureLimit=1000, totalPartialFailureLimit=20, successiveCompleteFailures=1, totalCompleteFailures=1, successivePartialFailures=0, totalPartialFailures=0)
Backoff before next attempt: 10 seconds
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.