Request Params without "?"

caquinhodomato · April 19, 2022, 5:42pm

Hello.

So, I am working with an API that does not use the “?” before the request params.

For example:

Instead of this:
https://api.rd.services/platform/contacts/?email=example@gmail.com

It works like this:
https://api.rd.services/platform/contacts/email:example@gmail.com

But if i use the request_params method, it always returns a ? before the email. How can I workaround this?

marcosmarxm · April 20, 2022, 4:19pm

The ? is added during the prepare_request process.
This is made during the read_records function where the method _create_prepared_request builds the request.

github.com

airbytehq/airbyte/blob/5164e12fc43c9856b2008932f793ecd19733d0ff/airbyte-cdk/python/airbyte_cdk/sources/streams/http/http.py#L251-L265

      
        
            def _create_prepared_request(
                self, path: str, headers: Mapping = None, params: Mapping = None, json: Any = None, data: Any = None
            ) -> requests.PreparedRequest:
                args = {"method": self.http_method, "url": urljoin(self.url_base, path), "headers": headers, "params": params}
                if self.http_method.upper() in BODY_REQUEST_METHODS:
                    if json and data:
                        raise RequestBodyException(
                            "At the same time only one of the 'request_body_data' and 'request_body_json' functions can return data"
                        )
                    elif json:
                        args["json"] = json
                    elif data:
                        args["data"] = data
            
            
    return self._session.prepare_request(requests.Request(**args))

I think you can overwrite the method to handle the ?, but you need to check the Request library how to do it.
Maybe if you join the url + params before calling the prepare_request would works for you:

github.com

psf/requests/blob/2d5517682b3b38547634d153cea43d48fbc8cdb5/requests/models.py#L365-L385

      
        
            def prepare_url(self, url, params):
                """Prepares the given HTTP URL."""
                #: Accept objects that have string representations.
                #: We're unable to blindly call unicode/str functions
                #: as this will include the bytestring indicator (b'')
                #: on python 3.x.
                #: https://github.com/psf/requests/pull/2238
                if isinstance(url, bytes):
                    url = url.decode('utf8')
                else:
                    url = str(url)
            
            
    # Remove leading whitespaces from url
                url = url.lstrip()
            
            
    # Don't do any URL preparation for non-HTTP schemes like `mailto`,
                # `data` etc to work around exceptions from `url_parse`, which
                # handles RFC 3986 only.
                if ':' in url and not url.lower().startswith('http'):
                    self.url = url
                    return

See lines 383-385

marcosmarxm · July 13, 2022, 12:00am

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.

Topic		Replies	Views
Request Params with different structuring Connector Development	5	895	July 14, 2022
Request params encryption Connector Questions & Issues connectors , connector-development	1	350	July 31, 2022
Params without encryption Connector Development	2	203	March 6, 2023
Response not in JSON format when testing API connector Connector Questions configuration , connector , question , json , api-connector	0	24	May 14, 2024
Trouble with User Input values in custom connector for API data extraction Connector Development airbyte-cloud , connector , question , custom-connector , api-data-extraction	2	27	June 23, 2024

Request Params without "?"

Related topics