Custom Connector Builder Issue with Incremental Streams

Summary

The user is facing issues with a custom connector built using Connector Builder, where the synchronization process sometimes takes longer than expected. They are specifically concerned about Incremental Streams and seek advice on debugging the configuration.


Question

Hi, I built a custom connector using Builder, but sometimes it works successfully, and sometimes it keeps running and synchronising data for more than 28GB and running for more than 10 hours. After that, I have to cancel it. Usually, it finishes within 30 minutes. The connector runs every 3 hours, and I am not expecting this amount of data to be updated quickly. Has anyone experienced an issue in Custom Builder for Incremental Streams or something similar? How should I debug that? Logs are not very helpful; it just keeps adding this log.
Attempting to read from queue namespace: {{stream_name}}
How can I check or debug what happens if anything is wrong with the builder stream configuration?



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["custom-connector", "builder", "incremental-streams", "debugging", "configuration"]