Summary
The user is facing a ‘Broken pipe’ error while moving data from MySQL to Redshift using Airbyte on AWS EKS. The error is related to a SocketException and IllegalStateException in the log.
Question
Hi everyone I need help with a MySQL (3.3.20v) -> Redshift (2.4.3v) connection, I’m running Airbyte 0.50.35v on AWS EKS.
I’m only moving 2 tables, 1 is tiny (500 rows) and the other one has 20+ million rows. The following is the tail of the log showing the error:
java.net.SocketException: Broken pipe
at sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:413) ~[?:?]
at sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:433) ~[?:?]
at sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:812) ~[?:?]
at java.net.Socket$SocketOutputStream.write(Socket.java:1120) ~[?:?]
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:313) ~[?:?]
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:409) ~[?:?]
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:414) ~[?:?]
at sun.nio.cs.StreamEncoder.lockedFlush(StreamEncoder.java:218) ~[?:?]
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:205) ~[?:?]
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:263) ~[?:?]
at java.io.BufferedWriter.implFlush(BufferedWriter.java:372) ~[?:?]
at java.io.BufferedWriter.flush(BufferedWriter.java:359) ~[?:?]
at io.airbyte.workers.internal.DefaultAirbyteMessageBufferedWriter.flush(DefaultAirbyteMessageBufferedWriter.java:31) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at io.airbyte.workers.internal.DefaultAirbyteDestination.notifyEndOfInputWithNoTimeoutMonitor(DefaultAirbyteDestination.java:140) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at io.airbyte.workers.internal.DefaultAirbyteDestination.notifyEndOfInput(DefaultAirbyteDestination.java:133) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at io.airbyte.workers.general.BufferedReplicationWorker.writeToDestination(BufferedReplicationWorker.java:442) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithTimeout$5(BufferedReplicationWorker.java:256) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2024-04-24 16:04:19 replication-orchestrator > writeToDestination: done. (forDest.isDone:false, isDestRunning:true)
2024-04-24 16:04:19 replication-orchestrator > Attempt 0 to update stream status incomplete materialbank:sdg_leads_report
2024-04-24 16:04:19 replication-orchestrator > readFromDestination: exception caught
java.lang.IllegalStateException: Destination process is still alive, cannot retrieve exit value.
at com.google.common.base.Preconditions.checkState(Preconditions.java:502) ~[guava-31.1-jre.jar:?]
at io.airbyte.workers.internal.DefaultAirbyteDestination.getExitValue(DefaultAirbyteDestination.java:191) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at io.airbyte.workers.general.BufferedReplicationWorker.readFromDestination(BufferedReplicationWorker.java:476) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:228) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2024-04-24 16:04:19 replication-orchestrator > readFromDestination: done. (writeToDestFailed:true, dest.isFinished:false)
2024-04-24 16:04:19 replication-orchestrator > Attempt 0 to update stream status incomplete materialbank:sdg_leads_report
2024-04-24 16:04:19 replication-orchestrator > processMessage: done. (fromSource.isDone:false, forDest.isClosed:true)
2024-04-24 16:04:19 replication-orchestrator > Attempt 0 to update stream status incomplete materialbank:sdg_leads_report_storage
2024-04-24 16:04:19 replication-orchestrator > readFromSource: exception caught
java.lang.IllegalStateException: Source process is still alive, cannot retrieve exit value.
at com.google.common.base.Preconditions.checkState(Preconditions.java:502) ~[guava-31.1-jre.jar:?]
at io.airbyte.workers.internal.DefaultAirbyteSource.getExitValue(DefaultAirbyteSource.java:127) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(BufferedReplicationWorker.java:362) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithHeartbeatCheck$3(BufferedReplicationWorker.java:235) ~[io.airbyte-airbyte-commons-worker-0.50.35.jar:?]
at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1589) ~[?:?]
2024-04-24 16:04:19 replication-orchestrator > readFromSource: done. (source.isFinished:false, fromSource.isClosed:true)
2024-04-24 16:04:19 replication-orchestrator > Attempt 0 to update stream status incomplete materialbank:sdg_leads_report
2024-04-24 16:04:19 replication-orchestrator > Attempt 0 to update stream status incomplete materialbank:sdg_leads_report_storage
2024-04-24 16:04:19 replication-orchestrator > Attempt 0 to update stream status incomplete materialbank:sdg_leads_report_storage
2024-04-24 16:04:22 replication-orchestrator > (pod: airbyte / destination-redshift-write-961-1-dbgeq) - Closed all resources for pod
2024-04-24 16:05:22 replication-orchestrator > (pod: airbyte / source-mysql-read-961-1-syorg) - Destroying Kube process.
2024-04-24 16:05:22 replication-orchestrator > (pod: airbyte / source-mysql-read-961-1-syorg) - Closed all resources for pod
2024-04-24 16:05:22 replication-orchestrator > airbyte-source gobbler IOException: Socket closed. Typically happens when cancelling a job.
2024-04-24 16:05:22 replication-orchestrator > (pod: airbyte / source-mysql-read-961-1-syorg) - Destroyed Kube process.```
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1713975246299249) if you want to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
["mysql", "redshift", "airbyte", "aws-eks", "broken-pipe", "socketexception", "illegalstateexception"]
</sub>