Custom Connector Incremental Sync Date Range Issue


The user is facing an issue with a custom connector where the DatetimeBasedCursor class truncates the end_datetime parameter at the current datetime instead of using the specified date range. They are looking for a way to achieve the desired behavior or considering modifying the code to remove the end date limitation.


I’m building a custom connector for an API that supports start_date and end_date parameters. With every sync I’d like to query a static date range +/- 2 weeks (4 weeks in total) and deduplicate objects based on an ID. I enabled Incremental sync as DatetimeBasedCursor and set the start_datetime to {{ now_utc() - duration('P14D') }} and end_datetime to {{ now_utc() + duration('P14D') }} . However the connector does not respect this and keeps using the current date as the end date. It looks like DatetimeBasedCursor class always truncates the end_datetime parameter at current datetime:| Is there a way to achieve this behaviour? Or would be the process of changing the code to remove the end_date limitation?

A similar question was posted here:

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["custom-connector", "incremental-sync", "datetimebasedcursor", "end-date", "date-range", "code-modification"]