I’m developing a new connector in Python for one of our client’s REST API. We authenticate via OAuth2 so I’ve extended Airbyte’s
airbyte_cdk.sources.streams.http.requests_native_auth.Oauth2Authenticator and had to re-implement
refresh_access_token() to suit my needs and to pass custom headers needed by the API.
When syncing, I sometimes get HTTP 401 errors after a custom backoff handling HTTP 408.
def should_retry(self, response: requests.Response) -> bool: return response.status_code in [401, 408] or super().should_retry(response) def backoff_time(self, response: requests.Response) -> Optional[float]: if response.status_code == 401: return 10 if response.status_code == 408: return randint(120, 240) return super().backoff_time(response)
When retrying for a HTTP 401, how can I reset the requests authentication header?
Token is set to expire 1 hours after being issued. The problem is that I can get a backoff at, say 58 minutes of the token life’s (2 minutes remaining), and when the backoff retries, the token is no longer valid (60+ minutes).
I want to recall the
Oauth2Authenticator magic method so that it can reset get a new access token:
def __call__(self, request): request.headers.update(self.get_auth_header()) return request