Cursor Pagination Issue with Opsgenie Alert API Connector

Slack Knowledge Base Q&A Connector Development
, , , ,
1

Summary

User is experiencing an issue with cursor pagination in the Opsgenie Alert API connector, where the connector enters a loop without logs when the ‘paging’ key is absent in the response. The provided YAML configuration details the stream setup, including the paginator and stop condition.


Question

Hi! I’m trying to run cursor pagination without success. The stop condition occurs when the response data does not have a “paging” key. However, the connector is getting stuck in a loop without providing any logs on what is happening. Does anyone know what might be the cause?

For reference, this is the Opsgenie Alert API, and here is the yaml file for the stream:

      type: DeclarativeStream
      name: AlertNotes
      primary_key:
        - offset
        - alert_id
      retriever:
        type: SimpleRetriever
        requester:
          $ref: "#/definitions/base_requester"
          path: v2/alerts/{{ stream_partition['id'] }}/notes
          http_method: GET
          request_parameters:
            limit: "100"
            order: desc
            identifierType: id
          error_handler:
            type: CompositeErrorHandler
            error_handlers:
              - type: DefaultErrorHandler
                max_retries: 10
                backoff_strategies:
                  - type: WaitUntilTimeFromHeader
                    header: X-RateLimit-Period-In-Sec
              - type: DefaultErrorHandler
        record_selector:
          type: RecordSelector
          extractor:
            type: DpathExtractor
            field_path:
              - data
        paginator:
          type: DefaultPaginator
          page_token_option:
            type: RequestPath
          pagination_strategy:
            type: CursorPagination
            cursor_value: "{{ response.get(\"paging\", {}).get(\"next\", {}) }}"
            stop_condition: "{{ not response.get(\"paging\", {}).get(\"next\", {}) }}"
        partition_router:
          type: SubstreamPartitionRouter
          parent_stream_configs:
            - type: ParentStreamConfig
              parent_key: id
              partition_field: id
              stream:
                $ref: "#/definitions/streams/Alerts"
      transformations:
        - type: AddFields
          fields:
            - path:
                - alert_id
              value: "{{ stream_partition['id'] }}"
      schema_loader:
        type: InlineSchemaLoader
        schema:
          $ref: "#/schemas/AlertNotes"```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C027KKE4BCZ/p1732275275565929) if you want
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
['cursor-pagination', 'opsgenie-alert-api', 'yaml-configuration', 'paginator', 'stop-condition']
</sub>
2

Have you tried to use existing https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-opsgenie/manifest.yaml|source-opsgenie/manifest.yaml as a template or you wrote your YAML from scratch?

3

Yes, I have tried both approaches and they get the same error.