Random 401 errors when the job is running for a large amount of time

Hello Team,
I am developing a custom source connector for Amazon Ads API, which will be a long running job. I am almost at the finish line, but I have an issue. After the job runs for a quite long time, I receive 401 errors from Amazon, however the job runs for more than an hour and the token is refreshed accordingly. However, sometimes I get this 401 error.
I tried to hard refresh the token in should_retry function: tokens: Tuple[str, int] = self.authenticator.refresh_access_token(), but it fails with the following error:

File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 294, in _send
    if self.should_retry(response):
  File "/airbyte/integration_code/source_amazon_ads/streams/common.py", line 292, in should_retry
    tokens: Tuple[str, int] = self.authenticator.refresh_access_token()
AttributeError: 'NoAuth' object has no attribute 'refresh_access_token'

I do not uderstand why the authenticator property is of type NoAuth, because when I initialize the stream in source, I pass the Oauth authenticator. Could someone help or point into the right direction?

@vcalmic did you check Amazon Ads API documentation explaining the reason of 401 errors? Maybe this can help you without doing random changes in your code.

@marcosmarxm Yes. The 401 error is thrown when the token is expired.

Can I recommend to you check the Source Kyriba, this sources does what you’re trying to achieve. Check the should_retry function: