Summary
User reports intermittent performance issues with a custom connector built using Connector Builder, where synchronization times exceed expected durations and data volume. Seeks advice on debugging incremental streams and stream configuration.
Question
Hi, I built a custom connector using Builder, but sometimes it works successfully, and sometimes it keeps running and synchronising data for more than 28GB and running for more than 10 hours. After that, I have to cancel it. Usually, it finishes within 30 minutes. The connector runs every 3 hours, and I am not expecting this amount of data to be updated quickly. Has anyone experienced an issue in Custom Builder for Incremental Streams or something similar? How should I debug that? Logs are not very helpful; it just keeps adding this log.
Attempting to read from queue namespace: {{stream_name}}
How can I check or debug what happens if anything is wrong with the builder stream configuration?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.