Summary
Custom connector built using Builder is experiencing long run times and synchronizing large amounts of data unexpectedly. Looking for guidance on debugging the issue related to Incremental Streams configuration.
Question
Hi, I built a custom connector using Builder, but sometimes it works successfully, and sometimes it keeps running and synchronising data for more than 28GB and running for more than 10hours. Usually, it finishes within 30 minutes. The connector runs every 3 hours, and I am not expecting this amount of data to be updated quickly. Has anyone experienced an issue in Custom Builder for Incremental Streams? How should I debug that? Logs are not much helping, it just keep adding this log
Attempting to read from queue namespace: {{stream_name}}
How can I check or debug what is happening if anything is wrong in the builder stream configuration?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.