Summary
The user encountered an error in the connector while trying to use low code CDK with a JSON input. The error message indicates a failure due to an unknown error. The user is seeking help to understand the problem.
Question
Hi everyone, I wanted to use low code CDK and implemented the below yaml file:
and I want to specify the json file as an input, but I got the error:
Something went wrong in the connector. See the logs for more details. - Traceback (most recent call last):
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py”, line 135, in read
yield from self._read_stream(
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py”, line 231, in _read_stream
for record_data_or_message in record_iterator:
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/streams/core.py”, line 174, in read
for record_data_or_message in records:
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/declarative_stream.py”, line 128, in read_records
yield from self.retriever.read_records(self.get_json_schema(), stream_slice)
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py”, line 375, in read_records
for stream_data in self._read_pages(record_generator, self.state, _slice):
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py”, line 298, in _read_pages
response = self._fetch_next_page(stream_state, stream_slice, next_page_token)
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py”, line 484, in _fetch_next_page
return self.requester.send_request(
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/requesters/http_requester.py”, line 294, in send_request
request, response = self._http_client.send_request(
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http_client.py”, line 331, in send_request
response: requests.Response = self._send_with_retry(request=request, request_kwargs=request_kwargs, log_formatter=log_formatter)
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http_client.py”, line 209, in _send_with_retry
response = backoff_handler(user_backoff_handler)(request, request_kwargs, log_formatter=log_formatter) # type: ignore # mypy can’t infer that backoff_handler wraps _send
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/backoff/_sync.py”, line 105, in retry
ret = target(*args, **kwargs)
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/backoff/_sync.py”, line 105, in retry
ret = target(*args, **kwargs)
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http_client.py”, line 291, in _send
raise UserDefinedBackoffException(
airbyte_cdk.sources.streams.http.exceptions.UserDefinedBackoffException: The request failed due to an unknown error.
ERROR
None - Traceback (most recent call last):
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/connector_builder/message_grouper.py”, line 302, in _read_stream
yield from AirbyteEntrypoint(source).read(source.spec(self.logger), config, configured_catalog, state)
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py”, line 166, in read
for message in self.source.read(self.logger, config, catalog, state):
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/manifest_declarative_source.py”, line 167, in read
yield from super().read(logger, config, catalog, state)
File “/home/airbyte/.pyenv/versions/3.9.19/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py”, line 185, in read
raise AirbyteTracedException(message=error_message, failure_type=FailureType.config_error)
airbyte_cdk.utils.traced_exception.AirbyteTracedException: NoneCan you please help me to underestand what the problem is?
version: 2.3.0
type: DeclarativeSource
check:
type: CheckStream
stream_names:
- location
definitions:
streams:
location:
type: DeclarativeStream
name: location
retriever:
type: SimpleRetriever
requester:
$ref: '#/definitions/base_requester'
path: >-
/location/{{stream_slice.city_name}}/{{stream_slice.street_name}}/{{stream_slice.house_number}}
http_method: GET
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path: []
schema_loader:
type: InlineSchemaLoader
schema:
$ref: '#/schemas/location'
slice_definer:
type: ListSliceDefiner
slice_values:
$ref: '{{ config.file_path }}'
field_names:
- city_name
- street_name
- house_number
base_requester:
type: HttpRequester
url_base: <http://172.21.0.13:8010>
streams:
- $ref: '#/definitions/streams/location'
spec:
type: Spec
connection_specification:
type: object
$schema: <http://json-schema.org/draft-07/schema#>
required:
- file_path
properties:
file_path:
type: string
order: 0
title: file_path
description: Path to the JSON file containing city, street, and house number data.
additionalProperties: true
metadata:
autoImportSchema:
location: true
schemas:
location:
type: object
$schema: <http://json-schema.org/schema#>
additionalProperties: true
properties:
lat:
type:
- number
- 'null'
lon:
type:
- number
- 'null'
name:
type:
- string
- 'null'```
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1723629078434479) if you want
to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
["error", "connector", "low-code-cdk", "json-input", "unknown-error"]
</sub>