Zuora Source "failed to fetch schema"

murat-cetinkaya · October 20, 2022, 6:30pm

Is this your first time deploying Airbyte?: Yes
OS Version / Instance: Linux (Amazon)
Memory / Disk: you can use something like 8Gb / 100 Gb
Deployment: Docker-Compose
Airbyte Version: 0.40.15
Source name/version: Zuora alpha 0.1.3
Destination name/version: Snowflake
Step: The issue is happening during sync
Description:

I’m getting failed to fetch schema error when

I hit refresh source schema
try to create a new connection

after upgraded to v0.40.15 from v0.40.14.

Thanks,
Murat

natalyjazzviolin · October 20, 2022, 7:34pm

Hey @murat, could you please attach any new logs you’re getting? I need more information to dig into this further!

murat-cetinkaya · October 20, 2022, 7:47pm

Hi @natalyjazzviolin ,
attached the logs and the screenshot.
Thanks.

server-logs.txt (991.0 KB)

natalyjazzviolin · October 20, 2022, 8:44pm

Are these the logs you are getting when performing the sync? I don’t see the error in these. Could you please go into the Status tab and get the logs from there? Like this:

murat-cetinkaya · October 20, 2022, 8:57pm

I don’t get any errors on syncs for the existing connection (attached the screenshot ob the Status tab).

I only get this error when I try to refresh the source schema or create a new connection. The logs I shared include the time when I tried to refresh the source schema.

And another point is that for existing connection I only see the enabled source objects (that I selected to ingest which are 4 in my case). There are normally more than 150 Zuora source objects listed in Replication tab but now I only see those 4 objects.

murat-cetinkaya · October 20, 2022, 9:49pm

I installed v0.40.14 again and saw it works almost fine. But later versions give the error. And I guess I could get the error log:

airbyte-temporal    | {"level":"info","ts":"2022-10-20T21:46:23.704Z","msg":"Get dynamic config","name":"system.visibilityArchivalState","value":"enabled","default-value":"enabled","logging-call-at":"config.go:79"}
airbyte-server      | 2022-10-20 21:46:24 INFO i.a.s.RequestLogger(filter):112 - REQ 172.19.0.2 POST 200 /api/v1/web_backend/connections/get - {"connectionId":"0924c93f-550e-4ba9-9641-e51f4651331c","withRefreshedCatalog":true}
airbyte-server      | Oct 20, 2022 9:46:24 PM org.glassfish.jersey.server.ServerRuntime$Responder writeResponse
airbyte-server      | SEVERE: An I/O error has occurred while writing a response message entity to the container output stream.
airbyte-server      | org.glassfish.jersey.server.internal.process.MappableException: org.eclipse.jetty.io.EofException
airbyte-server      | 	at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:67)
airbyte-server      | 	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
airbyte-server      | 	at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1116)
airbyte-server      | 	at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:638)
airbyte-server      | 	at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:371)
airbyte-server      | 	at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:361)
airbyte-server      | 	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
airbyte-server      | 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
airbyte-server      | 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
airbyte-server      | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
airbyte-server      | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
airbyte-server      | 	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
airbyte-server      | 	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
airbyte-server      | 	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)
airbyte-server      | 	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
airbyte-server      | 	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
airbyte-server      | 	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
airbyte-server      | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)
airbyte-server      | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)
airbyte-server      | 	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
airbyte-server      | 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)
airbyte-server      | 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:569)
airbyte-server      | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
airbyte-server      | 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)
airbyte-server      | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
airbyte-server      | 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)
airbyte-server      | 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
airbyte-server      | 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)
airbyte-server      | 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
airbyte-server      | 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
airbyte-server      | 	at org.eclipse.jetty.server.Server.handle(Server.java:501)
airbyte-server      | 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
airbyte-server      | 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
airbyte-server      | 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
airbyte-server      | 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
airbyte-server      | 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
airbyte-server      | 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
airbyte-server      | 	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
airbyte-server      | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
airbyte-server      | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
airbyte-server      | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
airbyte-server      | 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
airbyte-server      | 	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
airbyte-server      | 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
airbyte-server      | 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
airbyte-server      | 	at java.base/java.lang.Thread.run(Thread.java:1589)
airbyte-server      | Caused by: org.eclipse.jetty.io.EofException
airbyte-server      | 	at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
airbyte-server      | 	at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
airbyte-server      | 	at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:277)
airbyte-server      | 	at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:381)
airbyte-server      | 	at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:826)
airbyte-server      | 	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
airbyte-server      | 	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:223)
airbyte-server      | 	at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:544)
airbyte-server      | 	at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:838)
airbyte-server      | 	at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:910)
airbyte-server      | 	at org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:284)
airbyte-server      | 	at org.eclipse.jetty.server.HttpOutput.channelWrite(HttpOutput.java:268)
airbyte-server      | 	at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:833)
airbyte-server      | 	at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:301)
airbyte-server      | 	at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:200)
airbyte-server      | 	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:276)
airbyte-server      | 	at com.fasterxml.jackson.core.json.UTF8JsonGenerator._flushBuffer(UTF8JsonGenerator.java:2171)
airbyte-server      | 	at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeString(UTF8JsonGenerator.java:520)
airbyte-server      | 	at com.fasterxml.jackson.databind.node.TextNode.serialize(TextNode.java:151)
airbyte-server      | 	at com.fasterxml.jackson.databind.node.ArrayNode.serialize(ArrayNode.java:180)
airbyte-server      | 	at com.fasterxml.jackson.databind.node.ObjectNode.serialize(ObjectNode.java:328)
airbyte-server      | 	at com.fasterxml.jackson.databind.node.ObjectNode.serialize(ObjectNode.java:328)
airbyte-server      | 	at com.fasterxml.jackson.databind.node.ObjectNode.serialize(ObjectNode.java:328)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:39)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.std.SerializableSerializer.serialize(SerializableSerializer.java:20)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
airbyte-server      | 	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
airbyte-server      | 	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1518)
airbyte-server      | 	at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1007)
airbyte-server      | 	at org.glassfish.jersey.jackson.internal.jackson.jaxrs.base.ProviderBase.writeTo(ProviderBase.java:647)
airbyte-server      | 	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:242)
airbyte-server      | 	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:227)
airbyte-server      | 	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
airbyte-server      | 	at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:85)
airbyte-server      | 	at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:139)
airbyte-server      | 	at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:61)
airbyte-server      | 	... 45 more
airbyte-server      | Caused by: java.io.IOException: Broken pipe
airbyte-server      | 	at java.base/sun.nio.ch.FileDispatcherImpl.writev0(Native Method)
airbyte-server      | 	at java.base/sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:66)
airbyte-server      | 	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:226)
airbyte-server      | 	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:157)
airbyte-server      | 	at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:574)
airbyte-server      | 	at java.base/java.nio.channels.SocketChannel.write(SocketChannel.java:642)
airbyte-server      | 	at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:273)
airbyte-server      | 	... 95 more
airbyte-server      |

natalyjazzviolin · October 21, 2022, 3:02pm

Let me dig into this a little further and I’ll be back with some ideas soon!

sh4sh · October 25, 2022, 2:24pm

Request #2987 “I’m getting ‘failed to fetch sch…” was closed and merged into this request. Last comment in request #2987:

[via Slack] Hey, since there’s a forum thread on this issue we will continue the investigation there. Let us know if you notice anything else about the weird behaviour.

murat-cetinkaya · October 25, 2022, 2:29pm

Hi @natalyjazzviolin .
Could you find something?

natalyjazzviolin · November 7, 2022, 4:51pm

Hi Murat, I apologize for the delay - we’ve had a big influx of posts due to Hacktoberfest and are working on getting around to everything. Could you try allocating more resources to your Airbyte instance and/or the connector?

Here are the docs:
https://airbytehq.github.io/operator-guides/scaling-airbyte/

murat-cetinkaya · November 16, 2022, 3:38pm

Hi @natalyjazzviolin ,
No worries. I guessed you’d be very busy
I’d like to thank you for the Hactoberfest, lots of new features and connectors are coming

I changed the EC2 instance type to a one with more memory and CPU. I also set the following parameters but I get the same error

JOB_MAIN_CONTAINER_MEMORY_REQUEST=24g
JOB_MAIN_CONTAINER_MEMORY_LIMIT=25g

I also opened another thread in the below link with the logs if you want to see the logs:

Thank you,
Murat

murat-cetinkaya · November 16, 2022, 3:58pm

Hi again @natalyjazzviolin ,
I guess I found the reason.

github.com/airbytehq/airbyte

Proxy causes Gateway Timeout for dynamic schema discovery

opened 04:19PM - 10 Nov 22 UTC

henriblancke

type/bug community needs-triage team/use autoteam

## Environment - **Airbyte version**: `>=v0.40.16` - **OS Version / Instance*…*: Any - **Deployment**: Docker - **Source Connector and version**: `source-s3` - **Step where error happened**: Setup new connection -> discover dynamic schema ## Current Behavior When adding a new S3 connection, discovering the schema results in 504 Gateway Timeout. Trying again often resolves the issue. I wonder if the proxy's nginx default timeout should be increased to allow more time for dynamically fetching schemas like with the s3 connector (or any source where discovery can take +60 seconds) ## Expected Behavior The API returns the discovered schema. ## Steps to Reproduce 1. Setup source-s3 pointing to a bucket with larger files 2. Setup a connection to any destination 3. Wait for a non json response in the UI, in Inspect > Console it shows a 504

natalyjazzviolin · November 28, 2022, 6:00pm

We’re working on merging that PR! The build currently breaks when running CI/CD checks, but I’ll prioritize this for this week!

Topic		Replies	Views
SEVERE Error When Creating a new Zuora Source Connector Questions & Issues connectors	3	471	November 3, 2022
Failed to fetch schema(502 error) Connector Questions & Issues data-loading , connectors	1	288	March 23, 2023
Redshift source "Failed to fetch schema" Connector Questions & Issues connectors	2	391	September 12, 2022
Failed to fetch schema for GitHub Connector Connector Questions & Issues connectors	3	205	July 14, 2022
Failed to load schema in discovery_schema timeout in LoadBalancer Connector Questions & Issues getting-started	9	2194	September 30, 2022

Zuora Source "failed to fetch schema"

Related topics