Summary
The user is experiencing replication failures when attempting to transfer data from MySQL to Iceberg on S3 using Airbyte on AWS EC2. The error logs indicate issues with the destination connector, including a NoSuchMethodError related to logging and a non-zero exit code from the destination process.
Question
Hello everyone
I’m comming here because i have a problem when mysql to iceberg(s3) process(airbyte on aws ec2).
source connection, destination connection was no problem.
when i started connection process, i could see following errors.
i read these, but no luck, i couldn’t solve. can you help me on this issue?
},
"processFromDest" : {
"elapsedTimeInNanos" : 10772635,
"executionCount" : 1,
"avgExecTimeInNanos" : 1.0772635E7
},
"writeToDest" : {
"elapsedTimeInNanos" : 0,
"executionCount" : 0,
"avgExecTimeInNanos" : "NaN"
},
"readFromDest" : {
"elapsedTimeInNanos" : 10638369635,
"executionCount" : 1873,
"avgExecTimeInNanos" : 5679855.651361452
}
}
}
2024-01-16 06:39:35 platform > failures: [ {
"failureOrigin" : "destination",
"failureType" : "system_error",
"internalMessage" : "java.lang.NoSuchMethodError: org.apache.logging.slf4j.Log4jLoggerFactory: method 'void <init>()' not found",
"externalMessage" : "Something went wrong in the connector. See the logs for more details.",
"metadata" : {
"attemptNumber" : 0,
"jobId" : 1,
"from_trace_message" : true,
"connector_command" : "write"
},
"stacktrace" : "java.lang.NoSuchMethodError: org.apache.logging.slf4j.Log4jLoggerFactory: method 'void <init>()' not found\n\tat org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:53)\n\tat org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:41)\n\tat org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j2(Logging.scala:232)\n\tat org.apache.spark.internal.Logging.initializeLogging(Logging.scala:129)\n\tat org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:115)\n\tat org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:109)\n\tat org.apache.spark.SparkContext.initializeLogIfNecessary(SparkContext.scala:84)\n\tat org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:106)\n\tat org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)\n\tat org.apache.spark.SparkContext.initializeLogIfNecessary(SparkContext.scala:84)\n\tat org.apache.spark.internal.Logging.log(Logging.scala:53)\n\tat org.apache.spark.internal.Logging.log$(Logging.scala:51)\n\tat org.apache.spark.SparkContext.log(SparkContext.scala:84)\n\tat org.apache.spark.internal.Logging.logInfo(Logging.scala:61)\n\tat org.apache.spark.internal.Logging.logInfo$(Logging.scala:60)\n\tat org.apache.spark.SparkContext.logInfo(SparkContext.scala:84)\n\tat org.apache.spark.SparkContext.<init>(SparkContext.scala:195)\n\tat org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2714)\n\tat org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:953)\n\tat scala.Option.getOrElse(Option.scala:201)\n\tat org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:947)\n\tat io.airbyte.integrations.destination.iceberg.IcebergDestination.getConsumer(IcebergDestination.java:85)\n\tat io.airbyte.cdk.integrations.base.Destination.getSerializedMessageConsumer(Destination.java:54)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:186)\n\tat io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.java:125)\n\tat io.airbyte.integrations.destination.iceberg.IcebergDestination.main(IcebergDestination.java:42)\n",
"timestamp" : 1705387173918
}, {
"failureOrigin" : "destination",
"internalMessage" : "Destination process exited with non-zero exit code 1",
"externalMessage" : "Something went wrong within the destination connector",
"metadata" : {
"attemptNumber" : 0,
"jobId" : 1,
"connector_command" : "write"
},
"stacktrace" : "io.airbyte.workers.internal.exception.DestinationException: Destination process exited with non-zero exit code 1\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromDestination(BufferedReplicationWorker.java:477)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:228)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n",
"timestamp" : 1705387174349
}, {
"failureOrigin" : "source",
"internalMessage" : "Source process read attempt failed",
"externalMessage" : "Something went wrong within the source connector",
"metadata" : {
"attemptNumber" : 0,
"jobId" : 1,
"connector_command" : "read"
},
"stacktrace" : "io.airbyte.workers.internal.exception.SourceException: Source process read attempt failed\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(BufferedReplicationWorker.java:376)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithHeartbeatCheck$3(BufferedReplicationWorker.java:235)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: java.lang.IllegalStateException: Source process is still alive, cannot retrieve exit value.\n\tat com.google.common.base.Preconditions.checkState(Preconditions.java:502)\n\tat io.airbyte.workers.internal.DefaultAirbyteSource.getExitValue(DefaultAirbyteSource.java:127)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(BufferedReplicationWorker.java:362)\n\t... 5 more\n",
"timestamp" : 1705387174824
}, {
"failureOrigin" : "replication",
"internalMessage" : "io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.",
"externalMessage" : "Something went wrong during replication",
"metadata" : {
"attemptNumber" : 0,
"jobId" : 1
},
"stacktrace" : "java.lang.RuntimeException: io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.\n\tat io.airbyte.workers.general.BufferedReplicationWorker$CloseableWithTimeout.lambda$close$0(BufferedReplicationWorker.java:522)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithTimeout$5(BufferedReplicationWorker.java:256)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.\n\tat io.airbyte.workers.internal.DefaultAirbyteDestination.close(DefaultAirbyteDestination.java:161)\n\tat io.airbyte.workers.general.BufferedReplicationWorker$CloseableWithTimeout.lambda$close$0(BufferedReplicationWorker.java:520)\n\t... 5 more\n",
"timestamp" : 1705387174826
} ]
2024-01-16 06:39:35 platform >
2024-01-16 06:39:35 platform > ----- END REPLICATION -----
2024-01-16 06:39:35 platform >
2024-01-16 06:39:35 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=10, successivePartialFailureLimit=1000, totalPartialFailureLimit=10, successiveCompleteFailures=1, totalCompleteFailures=1, successivePartialFailures=0, totalPartialFailures=0)
Backoff before next attempt: 10 seconds```
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1705387599268449) if you want
to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
['mysql', 'iceberg', 's3', 'replication', 'airbyte', 'aws', 'error', 'connector']
</sub>