- Is this your first time deploying Airbyte?: Yes
- OS Version / Instance: centos7
- Memory / Disk: 48Gb / 200Gb
- Deployment: Docker
- Airbyte Version: latest
- Source name/version: oralce 11g
- Destination name/version: elasticsearch 7.10.2
- Step: during sync
- Description:
2022-09-14 12:20:36 e[43mdestinatione[0m > 2022-09-14 12:20:36 e[1;31mERRORe[m i.a.i.b.FailureTrackingAirbyteMessageConsumer(accept):52 - Exception while accepting message
2022-09-14 12:20:36 e[43mdestinatione[0m > java.lang.Exception: failed to write bulk records: {}
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.destination.elasticsearch.ElasticsearchAirbyteMessageConsumerFactory.lambda$recordWriterFunction$3(ElasticsearchAirbyteMessageConsumerFactory.java:94) ~[io.airbyte.airbyte-integrations.connectors-destination-elasticsearch-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.destination.record_buffer.InMemoryRecordBufferingStrategy.lambda$flushAll$1(InMemoryRecordBufferingStrategy.java:88) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.destination.record_buffer.InMemoryRecordBufferingStrategy.flushAll(InMemoryRecordBufferingStrategy.java:84) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.destination.record_buffer.InMemoryRecordBufferingStrategy.addRecord(InMemoryRecordBufferingStrategy.java:64) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer.acceptTracked(BufferedStreamConsumer.java:138) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.accept(FailureTrackingAirbyteMessageConsumer.java:50) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.IntegrationRunner.consumeWriteStream(IntegrationRunner.java:194) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.IntegrationRunner.lambda$runConsumer$4(IntegrationRunner.java:203) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.IntegrationRunner.watchForOrphanThreads(IntegrationRunner.java:232) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.IntegrationRunner.runConsumer(IntegrationRunner.java:202) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.IntegrationRunner.lambda$runInternal$1(IntegrationRunner.java:165) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) [io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:38) [io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:165) [io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:107) [io.airbyte.airbyte-integrations.bases-base-java-0.39.35-alpha.jar:?]
2022-09-14 12:20:36 e[43mdestinatione[0m > at io.airbyte.integrations.destination.elasticsearch.ElasticsearchDestination.main(ElasticsearchDestination.java:30)
logs-8.txt (3.1 MB)
Hi @Allen_Tseng,
Thanks for your post and welcome to the community. I saw your message on Slack, hopefully we can help get this resolved for you.
I need a little bit more info though. Firstly, what version connectors are you running for Oracle and Elasticsearch? And secondly, would it be possible for you to post the full sync logs? I’m not really seeing the error in the log messages you posted. Let me know if this is reasonable.
Try upgrading all of the connectors to their latest version and re-rerunning the sync.
Hey @Allen_Tseng,
Sorry that the sync isn’t successfully running. Based on the logs, are you using Airbyte version 0.39.5? If so, try upgrading the Airbyte version.
Furthermore, it looks like you are syncing a lot of tables from Oracle. Have you tried splitting this one big sync into multiple parallel syncs? Let me know if this is reasonable.