Incremental Sync Parameter Injection Issue

Summary

The user is experiencing an issue with the connector not correctly injecting incremental sync parameters into the API request URL, resulting in an incorrect URL format for incremental syncs despite multiple configuration attempts.


Question

Hi Team,

My connector is not correctly injecting incremental sync parameters into the API request URL. Despite multiple configuration attempts, the URL is not being constructed as expected for incremental syncs.

Current Behavior
The current URL being generated looks like this:
<https://api.us.nylas.com/v3/grants/2dd7fd4d-75da-4c23-8490-66a2fd26d4af/messages?search_query_native=220924800>

Expected Behavior
The expected URL should look like this:
<https://api.us.nylas.com/v3/grants/2dd7fd4d-75da-4c23-8490-66a2fd26d4af/messages?search_query_native=from%3A+%22%7Bvisa.com%7D%22+after%3A+1234567890>

Where 1234567890 is the Unix timestamp for the start of the current slice.

below is the Incremental sync which I have tried. Could someone help me correctly inject into ‘after’ and from,to,cc,bcc of ‘search_query_native’ query parameter

incremental_sync:
type: DatetimeBasedCursor
parameters:
- name: search_query_native
value: >
from: “{{ stream_partition[‘domains’] }}” OR
to: “{{ stream_partition[‘domains’] }}” OR
cc: “{{ stream_partition[‘domains’] }}” OR
bcc: “{{ stream_partition[‘domains’] }}”
after: {{ stream_interval.start_time }}
cursor_field: date
start_datetime:
type: MinMaxDatetime
datetime: “{{ config[‘start_date’] }}”
datetime_format: “%s”
datetime_format: “%s”
start_time_option:
type: RequestOption
field_name: search_query_native
inject_into: request_parameter
cursor_datetime_formats:
- “%s”



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['incremental-sync', 'api-request-url', 'parameter-injection', 'datetime-based-cursor']

<@U02TQLBLDU4> have you seen this already? I think I’ve seen a contribution to address this — am I tripping?

<@U069EMNRPA4> I couldn’t resolve this after trying multiple approaches.
It is using global stream state date instead stream specific date. The following is the Connection state

{
“streamDescriptor”: {
“name”: “Message_info”
},
“streamState”: {
“state”: {
“date”: 1731613475
},
“states”: [
{
“cursor”: {
“date”: 1696364137
},
“partition”: {
“domains”: “{http://capitalone.com|capitalone.com}”,
“grant_id”: “36df7ae1-f87e-4316-8c89-85af3548a71d”,
“parent_slice”: {}
}
}

<@U02TQLBLDU4> have you seen this already? I think I’ve seen a contribution to address this — am I tripping?

<@U069EMNRPA4> I couldn’t resolve this after trying multiple approaches.
It is using global stream state date instead stream specific date. The following is the Connection state

{
“streamDescriptor”: {
“name”: “Message_info”
},
“streamState”: {
“state”: {
“date”: 1731613475
},
“states”: [
{
“cursor”: {
“date”: 1696364137
},
“partition”: {
“domains”: “{http://capitalone.com|capitalone.com}”,
“grant_id”: “36df7ae1-f87e-4316-8c89-85af3548a71d”,
“parent_slice”: {}
}
}