Issue with query param special characters auto-escaping in UI Builder/low code connector development

Summary

The user is facing an issue where special characters in query params are being auto-escaped in the UI Builder/low code connector development. They are looking for a way to prevent these characters from being escaped to ensure compatibility with the LinkedIn API.


Question

Hi everyone :wave:
Using the UI Builder/low code connector development, I am facing an issue regarding query param special characters auto-escaping
How can I force them not to be escaped ?
Example:
• I set owner: (sponsoredAccount:urn%3Ali%3AsponsoredAccount%3A506989960) , that leads to owner=%28sponsoredAccount%3Aurn%253Ali%253AsponsoredAccount%253A506989960%29 in the final query => not accepted by LinkedIn API
• I would like to keep ( and : and ) original values
Thanks :slightly_smiling_face:



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["ui-builder-low-code", "connector-development", "query-param", "special-characters", "auto-escaping", "linkedin-api"]

have you tried to move request parameters to path?

<@U05JENRCF7C> thanks it works like that (screen 1) but does not look very “best-practice friendly” :smile:
NB: I also tried using Inject value into outgoing HTTP Request but still auto-escape (screen 2) :confused:

and more complex issue for incremental sync where API call needs for query param like:
submittedAtTimeRange=(start:1711407600000,end:1711494000000)
looks not manageable by low code :disappointed:

proposed PR : https://github.com/airbytehq/airbyte/discussions/42042 :pray:
(still not be able to manage those cases via low code)

<@U05FB419LMR> what part of it doesn’t work with low-code? (I ask because I feel like I’ve helped someone set one up in the past even in builder that worked this way, just by cramming it into the start range parameter while enabling the end time parameter but not actually injecting it anywhere)

<@U035912NS77>, thanks for reaching! :slightly_smiling_face:
I did not succeed to manage incremental sync because of query param that must be passed like submittedAtTimeRange=(start:1711407600000,end:1711494000000) , which does not look manageable via low-code :confused:
for the second topic about special chars auto-escaping, the workaround is to put them in the path , but better place would be in the query param section (with possibly flag to disable auto-escaping)
if you have any suggestion to manage those cases, happy to discuss :pray:

the escaping should be fine as long as the value isn’t already escaped (meaning use the literal :, not %3A in your code so that it outputs it as only since-encoded—this goes for the urn: format too which should just be sponsoredAccount:urn:li:sponsoredAccount:506989960). They’ll accept the single-encoded value, but if you’re putting it in as %3A it’ll get double-encoded to %253A which they will NOT accept. You can see them accepting the URL-encoded values in their Postman examples:
https://www.postman.com/linkedin-developer-apis/linkedin-marketing-solutions-versioned-apis/request/17563548-a8d81123-687c-4f0f-809c-79b85386532a

With that said, it shouldn’t really encode the parentheses, but I double their system will have an issue with that (since they’re already decoding the string). When in doubt, test in Postman and confirm whether it’s an issue there

On the formatting of the dates, I believe you can just inject a query param for submittedAtTimeRange with a value based on jinja merges (IIRC for both start and end times to exist, you need both blocks but you’d only inject on one), like (start:{{ timestamp(stream_interval.start_time) * 1000 }},end:{{ timestamp(stream_interval.end_time) * 1000 }})

The added complexity here is that python doesn’t natively have a millisecond or microsecond timestamp, and IIRC you can’t operate on the native date object in Airbyte (where you could do datetime.microsecond/1000 or something). But in this case just multiplying the timestamp number of seconds by 1000 should be suitable.

Note, this jinja is just off the top of my head, but you’ll need to test it and make sure it plays nicely.

<@U035912NS77>, thanks for details
your API endpoint is the legacy one, please use base url https://api.linkedin.com/rest/leadForms to confirm the encoding issue with postman :pray:
to be more precise, the : expected right after sponsoredAccount can not be encoded otherwise returns 400 error
ex:

regarding incremental sync, not sure how to concretely inject variables you mentioned
do you have any documentation to provide ? :pray:
cf below failing example as I can not choose how to fill the query param value (just choice on key name)