Is it normal that "stream_state" not kept across sliced streams

Connector Development
1

I’m developing a connector for an API. I just have to set the following json in the body:

{

“all_documents_for_location”: “de”,

“size”: 100,

“offset”: 100,

“inserted_time”: {

“since”: “2023-07-10T00:00:00Z”,

“until”: “2023-07-10T00:10:00Z”

},

“order”: “asc”

}

  • where all_documents_for_location is where I set the country code to get docs from a country

  • size and offset are for pagination

In stream_slices, I iterate country codes, and so I’ll have separate sliced streams to download data from API with different country codes.

I also set up state properties to store the end time (i.e. until in the JSON body) for each sliced stream as the start time (i.e. since in the JSON body) in the next sync

It seems the first sliced stream works okay, but for the following ones, stream_state is always {} for all the CDK functions (i.e. request_body_json, read_records)

Is it normal for stream_state becomes empty after the 1st processed sliced stream? or did I do something wrong?

Any help is appreciated! Thank you!

Config:
{“start_date”: “2023-07-10T00:10:00Z”, “country”: [“jp”,“de”], “apikey”: “XXXXXX”}

Complete code:

2

and it seems if I add the following code, it works okay

def get_updated_state(self, current_stream_state: MutableMapping[str, Any], latest_record: Mapping[str, Any]) → Mapping[str, Any]:
return self._cursor_value

But to my knowledge, this method is already deprecated…should I still use this method?