Usage of `incremental_sync` yaml node in Full Refresh mode

Summary

Explanation of how to use incremental_sync yaml node in Full Refresh mode and how to implement an incremental stream with an updated_at cursor while initially running a full refresh of all objects.


Question

Hello everyone. Can anyone explain how the incremental_sync yaml node should be used in a Full Refresh mode? I’d expect that when the Full Refresh mode is used the incremental_sync yaml node is just ignored and the stream would just fetch all data using the defined pagination strategy without using incremental parameters. Aparently this seems to not to be the case and from what I observed the Full Refresh mode just doesn’t record the final stream state. That said, is there a way to implement an incremental stream (let’s say with updated_at cursor), but initially when a stream is created run a full refresh of all objects regardless when they were last updated?



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["incremental-sync", "full-refresh", "pagination-strategy", "stream-state", "updated-at-cursor"]