Summary
When using connector-builder to sync a parent-stream and sub-stream with incremental syncs, how to make the sub-stream incremental as well to pull data only for the partition values pulled fresh in the parent stream. Guidance on marking the sub-stream as incremental and defining the cursor is needed.
Question
I’m using connector-builder to sync a parent-stream and sub-stream. The parent stream has incremental syncs. How do I make the sub-stream “incremental” as well - meaning the sub-stream should only pull data for the partition values that were pulled fresh in parent stream? Do I need to mark the sub-stream as incremental as well? If so, not sure what the cursor etc would be. Any guidance is appreciated
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.