Request Params with different structuring

Connector Development
1

I am developing an connector which consumes data from an API wich uses parameters in a unusual way. Instead of using parameters like “id=521123&email=example@email”, it writes “id:12321&email:example@email”. How can I manage it in Airbyte?

2

Hi @caquinhodomato,
This indeed not a conventional way of separating key from value in a HTTP request.
You can try to make requests params return a string rather than a dict which can make the python requests module send the query string as it is.

e.g.:

def request_params(self,  stream_state: Mapping[str, Any], stream_slice: Mapping[str, Any] = None, next_page_token: Mapping[str, Any] = None,):
  #add whatever logic you need to build the dictionary  below
  params = {"id": 521123, "email": "example@email"}
  return "&".join([f"{k}:{v}" for k,v in params.items()])

Let me know if it works!

5

Hello @alafanechere .
This is a good solution, but I have found another problem with the API. It does not use the “?” before the parameters. For example, the correct URL for a request would be https://api.rd.services/platform/contacts/email:example@gmail.com , and not https://api.rd.services/platform/contacts/?email:example@gmail.com .

Do you know any workarounds with Airbyte?

6

Hi @caquinhodomato,
I would then not implement request_params and rather implement it in a path property:

    def path(
        self,
        *,
        stream_state: Mapping[str, Any] = None,
        stream_slice: Mapping[str, Any] = None,
        next_page_token: Mapping[str, Any] = None,
    ) -> str:
         params = {"id": 521123, "email": "example@email"}
         joined_params = "&".join([f"{k}:{v}" for k,v in params.items()])
         return f"platform/contacts/{joined_params}"

This is a hack and not CDK best practice of course :slight_smile:

1 Like
7

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.