Backfilling Daily Data from Cumulative API

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

User inquires about backfilling daily data using the connector builder for an API that only provides cumulative data. They seek a method to loop requests from a specified start date to achieve daily granularity.


Question

Using the connector builder is there a way to backfill daily data from an API that only returns cumulative data?
The API returns cumulative data from a start date and end date. I created a connector that streams daily data by setting the start date and end date with

  "endDate": "{{ day_delta(0, format='%Y-%m-%d') }}"```
However this does not allow me to backfill data beyond the date i start running the connector.
is there a way to loop the request from a start date on a daily granularity?

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1706037383610969) if you want
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
['connector-builder', 'backfill', 'cumulative-data', 'daily-granularity', 'api']
</sub>