Summary
The user is facing challenges with the Token Expiry Property Name field in the connector builder UI while working on a custom connector that uses OAuth 2.0 client credentials flow for authentication. They are unable to specify the correct token expiry format and encounter errors when trying to refresh the token. They seek advice on how to proceed with ensuring the connector authenticates correctly.
Question
Hi all – I’m building a custom connector that uses an OAuth 2.0 client credentials flow for authentication. I can get access tokens no problem, but the connector builder UI is forcing me to provide a valid value for ‘Token Expiry Property Name’.
• The only token expiry property I get is in the header of the API response – not the body.
• I do, however, get an “issued_at” property in the body – but it’s in milliseconds, not seconds, and therefore if I try to use that, it gives a “date out of range” error.
◦ It seems that there’s no possible way to use the Token Expiry Date Format field to tell Airbyte to expect milliseconds rather than seconds, which is default (i.e. that seems to only be for string-based date formatting, no epoch control)
• If there’s even a way I could tell it to refresh every 4 hours/14,400 seconds, I’d be fine with doing that, but putting 14400 in the token expiry property name field throws a key error (i.e. it searches for a field in the response body called ‘14400’)
Any suggestions on how to proceed to get this connector to use the token to authenticate correctly? It’s frustrating because it seems as though all is working properly, I’m just much too constrained on how to tell Airbyte when to request a new token/when to expect the current token will expire. I’m really hoping I don’t have to pivot to use Airbyte’s python CDK to build this connector because, outside of this step, my connector is completely done and tested as working properly (I confirmed by manually copying the refresh token returned by this above flow and treating it as a bearer authentication API, which worked correctly).
Thanks a ton!
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["oauth-2.0", "client-credentials-flow", "token-expiry", "connector-builder", "authentication"]
Based on the information provided in the extracts, it seems that you’re facing a challenge with the OAuth authentication method in Airbyte’s connector builder UI. The “Token Expiry Property Name” field is expecting the name of the property in the response that contains the expiry time of the access token. If not specified, the expiry property is interpreted as the number of seconds the access token will be valid source. If the expiry time is provided in the header of the API response and not in the body, it might not be possible to specify it in the connector builder UI. Similarly, if the “issued_at” property is in milliseconds, it might not be compatible with the default interpretation of the expiry property as seconds. The “Token Expiry Date Format” field is used to specify the format of the time to expiration datetime if the time is returned as a date-time string instead of seconds source. Unfortunately, based on the provided extracts, it doesn’t seem possible to specify a fixed refresh interval like every 4 hours/14,400 seconds in the token expiry property name field. If the API you’re working with uses other grant types like PKCE, it might not be possible to use the connector builder with OAuth authentication source. In this case, you might need to consider other options like using Airbyte’s Python CDK to build the connector. However, this would require additional work and might not be the most efficient solution if your connector is already complete and tested. I recommend checking out the Airbyte documentation and community forum for more information and potential solutions.
Hey coming back with more info around my use case – it appears that the lack of expires_in or any similar value in the response of the API body is by design.
The way the API devs intend for us to use the API is to first attempt an API call, then if we get a 4xx/Unauthorized error, then we go and attempt to refresh the token and try again. So, there is no truly predictable/reliable way to know how long a token will be valid for, by design, because they are session-based.
I wonder if it’d be possible, maybe in a future release, to make the token expiry date fields not required? (I know they technically aren’t required now, but only in the sense that it defaults to a value to use if you don’t provide one). If that were the case, and no value was provided for the token expiry date name, maybe the connector could then mimic this behavior, using the token it has until a certain error type is encountered, upon which case it refreshes the token then tries again. Just some suggestions of course.
Thanks again for your help and time!
The latest version of the Airbyte CDK can be used without the expires_in (if I’m not wrong) but this you’re going to be only able to use with the low-code framework as the new release was not updated in the connector builder yet.
Awesome <@U01MMSDJGC9>! Thanks a ton for that info.