Custom Connector Pagination Issue with Planning Center API

Slack Knowledge Base Q&A Connector Development
, , , ,
1

Summary

Developing a custom connector for Planning Center API using low-code interface, facing pagination issues. Question regarding dynamically adjusting rate limits based on response headers.


Question

Hello!
I’m using the low-code interface to develop a custom connector for <Planning Center API Documentation Center> and I am running into issues with pagination. In reading their documentation, I am wondering if the following conditions can be built within the low-code interface, or this would require a custom integration.

From the documentation

If you think your app may be affected by rate limits, it should be developed to adjust dynamically inspecting the values of X-PCO-API-Request-Rate-Limit and X-PCO-API-Request-Rate-Period in the HTTP headers. Your current count is discoverable in the X-PCO-API-Request-Rate-Count key. Requests that exceed the current limit will return an HTTP status 429. The number of seconds to wait until you can retry those requests is in the key Retry-After.
Response headers:

  "headers": {
    "Date": "Wed, 03 Apr 2024 15:56:23 GMT",
    "X-PCO-API-Auth-Method": "HTTPBasic",
    "X-PCO-API-Request-Rate-Count": "5",
    "X-PCO-API-Request-Rate-Limit": "100",
    "X-PCO-API-Request-Rate-Period": "20",
    "X-PCO-API-Processor": "ENG_6.3.0",
    "X-PCO-API-Processed-As-Version": "2019-10-18",
    "Vary": "Accept",
    "Cache-Control": "max-age=0, private, must-revalidate",
    "X-Runtime": "0.094593",
    "Strict-Transport-Security": "max-age=63072000; includeSubDomains"
  }```

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["custom-connector", "pagination-issue", "planning-center-api", "rate-limiting", "response-headers"]
2

it seems like the API would include a Retry-After header if the connector hits the rate limit. you could use a Wait time from header backoff strategy to extract the value and backoff

3

I attempted adding that and no luck.

Based on the documentation notes, does this configuration make sense?
(Sorry for the novice questions)

4

Those headers appear to be related to rate limits, not pagination. looking at <https://developer.planning.center/docs/#/apps/api/2019-02-13/vertices/organization#url-parameters|the docs>, I think you’ll need to use the per_page and offsetquery parameters to paginate.

5

> Ah yes, thanks. I have those parameters already set in the query_parameters config, I think im just confused as to where to configure the call to inspect the X-PCO-API-Request-Rate-Limit and X-PCO-API-Request- headers, or if that is even necessary.

6

there’s no way to throttle the connector based off these headers right now. the best you can do is use the Retry-After header when hitting a HTTP 429

7

<@U06FVATD45N> but that :upvote: said, we have throttling / rate limit fallback behavior in Connector Builder on our roadmap. Usually that means we’re considering building it within the next 2-3 months. Disclaimer: not a commitment or a promise. Check back, we might have it soon, but could go either way. Priorities change /shrug