Handling rate limit response in connector builder for GitBook

Summary

Exploring how to handle rate limit responses in connector builder for GitBook and mapping X-RateLimit-Reset header to GitBook’s retry mechanism


Question

I am using connector builder to make a connector for a service that has a rate limit. When we reach the rate limit we can get a response like this one:

> X-RateLimit-Limit: 60
> X-RateLimit-Remaining: 0
> X-RateLimit-Reset: 1377013266

> {
>    "error": {
>        "code": 429
>        "message": "API rate limit exceeded. Please try again in 60 seconds"
>    }
> }```
`X-RateLimit-Reset` is the GMT time when the rate limit will reset

How do I tell GitBook to wait until that time to retry?

I found the `Backoff Strategy` which I think is the correct neighborhood. I think that takes a delay until instead of a wait until value. How do I map this header to what GB needs?

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C027KKE4BCZ/p1712588489707959) if you want to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["rate-limit", "connector-builder", "gitbook", "backoff-strategy", "x-ratelimit-reset"]
</sub>

This is the description for those header values:

X-RateLimit-Limit The maximum number of requests you’re permitted to make in the current rate limit window.

X-RateLimit-Remaining The number of requests remaining in the current rate limit window.

X-RateLimit-Reset The time at which the current rate limit window resets in UTC epoch seconds.

Maybe this?

https://docs.airbyte.com/connector-development/connector-builder-ui/error-handling/#wait-until-time-from-header

<@U04NSL0BPD0> did Wait until time from header work for you?

I am not sure how to test it. I added it here and when I click test I just get an error. How do I know if it works?

What’s the error you’re getting?

I get 429 back (rate limit) so I added the x-ratelimit-reset how do I know if it is parsing that correctly?

I mean the traceback there which should end in an error

Released it and ran it and it does not appear to be honoring the Header value

<@U047ANT3J84> any thoughts on this?

I think I have it working now.

I’m glad you got it working!