Failed to fetch schema

Connector Development
1

I am trying to build a custom connector using python, which has multiple streams, connector build was successful added it to airbyte GUI and source&destination all test cases passed, but after this, I am facing an error failed to fetch the schema

Ref:-
spec.json file:-"required": ["q"],
    "additionalProperties": false,
    "properties": {
      "q": {
        "title": "City",
        "type": "string",
        "pattern": "^[A-z 0-9  _\\-.,]+$",
        "examples": ["Tokyo", "istanbul", "New York", "London", "Paris", "Bangkok", "Beijing", "Mexico", "Mumbai",
                    "Delhi", "Hyderabad", "Bangalore", "Madrid", "Dubai", "Sydney", "Manchester", "Liverpool"],
        "description": "city data you requested from api"
      }
    }


source.json file:-
class Sports(MultiStreamWeatherApiStream):
    # Set this as a noop.
    primary_key = None
    def path(self, **kwargs) -> str:
        weather_list = self.weather_list
        # This defines the path to the endpoint that we want to hit.
        return f"/sports.json?q={weather_list}&rapidapi-key=*************"

class TimeZone(MultiStreamWeatherApiStream):
    # Set this as a noop.
    primary_key = None
    def path(self, **kwargs) -> str:
        weather_list = self.weather_list
        # This defines the path to the endpoint that we want to hit.
        return f"/timezone.json?q={weather_list}&rapidapi-key=***********************"

#source
class SourceMultiStreamWeatherApi(AbstractSource):
    def check_connection(self, logger, config) -> Tuple[bool, any]:
            input_list = config["q"]
            if input_list not in weather_list.Weather_List:
                return False, f"Input List {input_list} is invalid. Please check your spelling and input a valid City."
            else:
                return True, None
    def streams(self, config: Mapping[str, Any]) -> List[Stream]:
            return [Sports(weather_list=config["q"]),
                    TimeZone(weather_list=config["q"])]
2

Hey @Bharath,
Did you first try to run the discover schema command on your connector container?
There is probably an error in this part of your connector.

What is the output of:

docker run --rm -v $(pwd)/secrets:/secrets airbyte/<your-connector>:dev discover --config /secrets/config.json
5

After updating the config.json file then we re-ran the command given by you we get this error
docker run --rm -v %cd%/secrets:/secrets airbyte/source-multi-stream-weather-api:dev discover --config /secrets/config.json
this is the output:-

{"type": "LOG", "log": {"level": "FATAL", "message": "[Errno 2] No such file or directory: '/airbyte/integration_code/source_multi_stream_weather_api/schemas/sports.json'\nTraceback (most recent call last):\n File \"/airbyte/integration_code/main.py\", line 13, in <module>\n launch(source, sys.argv[1:])\n File \"/usr/local/lib/python3.7/site-packages/airbyte_cdk/entrypoint.py\", line 127, in launch\n for message in source_entrypoint.run(parsed_args):\n File \"/usr/local/lib/python3.7/site-packages/airbyte_cdk/entrypoint.py\", line 112, in run\n catalog = self.source.discover(self.logger, config)\n File \"/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py\", line 72, in discover\n streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]\n File \"/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/abstract_source.py\", line 72, in <listcomp>\n streams = [stream.as_airbyte_stream() for stream in self.streams(config=config)]\n File \"/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/core.py\", line 101, in as_airbyte_stream\n stream = AirbyteStream(name=self.name, json_schema=dict(self.get_json_schema()), supported_sync_modes=[SyncMode.full_refresh])\n File \"/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/streams/core.py\", line 98, in get_json_schema\n return ResourceSchemaLoader(package_name_from_class(self.__class__)).get_schema(self.name)\n File \"/usr/local/lib/python3.7/site-packages/airbyte_cdk/sources/utils/schema_helpers.py\", line 124, in get_schema\n raw_file = pkgutil.get_data(self.package_name, schema_filename)\n File \"/usr/local/lib/python3.7/pkgutil.py\", line 637, in get_data\n return loader.get_data(resource_name)\n File \"<frozen importlib._bootstrap_external>\", line 916, in get_data\nFileNotFoundError: [Errno 2] No such file or directory: '/airbyte/integration_code/source_multi_stream_weather_api/schemas/sports.json'"}}

And there is no integration_code folder in my airbyte folder

6

The /airbyte/integration_code refers to a directory on your docker image to which your connector is copied. Did you rebuild the image and confirm that you have a schemas/sports.json file in your source_multi_stream_weather_api folder?

7

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.