Error when running source.check() in pyairbyte for custom API connector

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

The user is encountering an error when running source.check() in pyairbyte for a custom API connector. The log file is empty and the error message indicates a system error.


Question

Hello All,
I’m building a custom api connector in pyairbyte. My yaml file looks like this:


type: DeclarativeSource

check:
  type: CheckStream
  stream_names:
    - mongo-test

definitions:
  streams:
    mongo-test:
      type: DeclarativeStream
      name: mongo-test
      retriever:
        type: SimpleRetriever
        requester:
          $ref: "#/definitions/base_requester"
          path: <https://test-api.videoland.com/subscribe/trusted-party/v1/entitlements>
          http_method: GET
          request_parameters:
            LastModifiedFrom: "2024-01-01"
            LastModifiedTo: "2024-08-01"
            limit: "5"
        record_selector:
          type: RecordSelector
          extractor:
            type: DpathExtractor
            field_path: []
        paginator:
          type: DefaultPaginator
          page_token_option:
            type: RequestOption
            inject_into: request_parameter
            field_name: cursor
          pagination_strategy:
            type: CursorPagination
            cursor_value: "{{ response['paging']['nextCursor'] }}"
            stop_condition: "{{ 'nextCursor' not in response['paging']}}"
  base_requester:
    type: HttpRequester
    url_base: <https://test-api.videoland.com/subscribe/trusted-party/v1/entitlements>
    authenticator:
      type: OAuthAuthenticator
      refresh_request_body: {}
      token_refresh_endpoint: <https://test-api.videoland.com/subscribe/identity-server/connect/token>
      grant_type: client_credentials
      client_id: "xxxx"
      client_secret: "xxxx"

streams:
  - $ref: "#/definitions/streams/mongo-test"```
when i do source.check() in pyairbyte
I'm getting following error -  raise exc.AirbyteConnectorFailedError(\nairbyte.exceptions.AirbyteConnectorFailedError: AirbyteConnectorFailedError: Connector failed.\n    Connector Name: 'source_mongo_api'\n    Log file: /var/folders/r8/2z5d0tfs0mj9csmchdg1hwcm0000gp/T/airbyte/logs/source_mongo_api/connector-log-01J5TM8PZDS7585M4D6TW6GK16.txt\n","failure_type":"system_error","stream_descriptor":null}}}

The log file is empty. Any help ?

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1724249976429299) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["custom-api-connector", "pyairbyte", "source.check()", "error", "empty-log-file"]
</sub>