Forbidden error via API when connecting from Python SDK

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

Forbidden error is encountered when trying to connect to the API from Python SDK, despite successful connections via UI. Various authentication methods have been attempted without success.


Question

Hey guys!
Getting forbidden error via API
• abctl simple launch for OSS
• UI is reachable with username pass
• all functionality in ui is good (successive connections syncs etc.)
• when I try to connect from python SDK with same username/pass getting Forbidden error
• also tried different ways for auth (token etc.) - same forbidden
• username/password looks ok since when I change them I’m getting unauthorized error.
here is full error

Cell In[92], line 1
----> 1 res = s.connections.get_connection(request=api.GetConnectionRequest(
      2     connection_id='cdd4bf7e-ed72-406a-a882-6d9a1f6adcd1',
      3 ))
      5 if res.connection_response is not None:
      6     # handle response
      7     pass

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/connections.py:173, in Connections.get_connection(self, request)
    171         raise errors.SDKError(f'unknown content-type received: {content_type}', http_res.status_code, http_res.text, http_res)
    172 elif http_res.status_code == 403 or http_res.status_code == 404 or http_res.status_code >= 400 and http_res.status_code < 500 or http_res.status_code >= 500 and http_res.status_code < 600:
--> 173     raise errors.SDKError('API error occurred', http_res.status_code, http_res.text, http_res)
    174 else:
    175     raise errors.SDKError('unknown status code received', http_res.status_code, http_res.text, http_res)

SDKError: API error occurred: Status 403
{"message":"Forbidden","_embedded":{"errors":[{"message":"Forbidden","_embedded":{},"_links":{}}]},"_links":{"self":{"href":"/api/v1/connections/cdd4bf7e-ed72-406a-a882-6d9a1f6adcd1","templated":false}}}```
and here is code to reproduce
```import airbyte_api
from airbyte_api import models, api
s = airbyte_api.AirbyteAPI(
    server_url='<http://localhost:8000/api/v1>',
    security=models.Security(
        basic_auth=models.SchemeBasicAuth(
            password='pass',
            username='name',
        ),
    ),
)
res = s.connections.get_connection(request=api.GetConnectionRequest(
    connection_id='some_id',
))```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1722874470501729) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["forbidden-error", "api", "python-sdk", "authentication", "ui-connection"]
</sub>
2

Can you try using the Application (client id and client secret)?

3 
    server_url="<https://localhost:8000>",
    security=models.Security(
        client_credentials=models.SchemeClientCredentials(
            client_id=client_id,
            client_secret=client_secret,
            TOKEN_URL="v1/applications/token",
        ),
    ),
)```
I was able to get using the API running with the following code.
4

<@U01MMSDJGC9>
still some problem

with code:

from airbyte_api import models, api

s = airbyte_api.AirbyteAPI(
    server_url="<http://localhost:8000/api/v1>",
    security=models.Security(
        client_credentials=models.SchemeClientCredentials(
            client_id='id',
            client_secret='secret',
            TOKEN_URL="applications/token",
        ),
    ),
)
res = s.connections.get_connection(request=api.GetConnectionRequest(
    connection_id='some_id',
))```
getting error
```Exception                                 Traceback (most recent call last)
Cell In[10], line 14
      2 from airbyte_api import models, api
      4 s = airbyte_api.AirbyteAPI(
      5     server_url="<http://localhost:8000/api/v1>",
      6     security=models.Security(
   (...)
     12     ),
     13 )
---&gt; 14 res = s.connections.get_connection(request=api.GetConnectionRequest(
     15     connection_id='some_id',
     16 ))

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/connections.py:149, in Connections.get_connection(self, request)
    147     _, e = self.sdk_configuration.get_hooks().after_error(AfterErrorContext(hook_ctx), None, e)
    148     if e is not None:
--&gt; 149         raise e
    151 if utils.match_status_codes(['403','404','4XX','5XX'], http_res.status_code):
    152     result, e = self.sdk_configuration.get_hooks().after_error(AfterErrorContext(hook_ctx), http_res, None)

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/connections.py:144, in Connections.get_connection(self, request)
    142 try:
    143     req = client.prepare_request(requests_http.Request('GET', url, params=query_params, headers=headers))
--&gt; 144     req = self.sdk_configuration.get_hooks().before_request(BeforeRequestContext(hook_ctx), req)
    145     http_res = client.send(req)
    146 except Exception as e:

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/_hooks/sdkhooks.py:41, in SDKHooks.before_request(self, hook_ctx, request)
     39 def before_request(self, hook_ctx: BeforeRequestContext, request: requests.PreparedRequest) -&gt; requests.PreparedRequest:
     40     for hook in self.before_request_hooks:
---&gt; 41         out = hook.before_request(hook_ctx, request)
     42         if isinstance(out, Exception):
     43             raise out

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/_hooks/clientcredentials.py:59, in ClientCredentialsHook.before_request(self, hook_ctx, request)
     55 session_key = self.get_session_key(
     56     credentials.client_id, credentials.client_secret)
     58 if session_key not in self.sessions or not self.has_required_scopes(self.sessions[session_key].scopes, hook_ctx.oauth2_scopes) or self.has_token_expired(self.sessions[session_key].expires_at):
---&gt; 59     sess = self.do_token_request(credentials, self.get_scopes(
     60         hook_ctx.oauth2_scopes, self.sessions.get(session_key)))
     62     self.sessions[session_key] = sess
     64 request.headers["Authorization"] = f"Bearer {self.sessions[session_key].token}"

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/_hooks/clientcredentials.py:122, in ClientCredentialsHook.do_token_request(self, credentials, scopes)
    119 response = <http://self.client.post|self.client.post>(token_url, data=payload)
    121 if response.status_code &lt; 200 or response.status_code &gt;= 300:
--&gt; 122     raise Exception(
    123         f"Unexpected status code {response.status_code} from token endpoint")
    125 response_data = response.json()
    127 if response_data.get("token_type") != "Bearer":

Exception: Unexpected status code 401 from token endpoint```
5

What version of the airbyte-api Python library are you using?

6

Try to copy my example in exact details. How I built the server url and TOKEN_URL

7

with exact your code I’m getting
SSLError: HTTPSConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /v1/applications/token (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1006)')))

8

going with http is same I provided

Exception                                 Traceback (most recent call last)
Cell In[14], line 35
      3 # s = airbyte_api.AirbyteAPI(
      4 #     server_url='<http://localhost:8000/api/v1>',
      5 #     security=models.Security(#client_credentials='''eyJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJhaXJieXRlLXNlcnZlciIsInN1YiI6IjAwMDAwMDAwLTAwMDAtMDAwMC0wMDAwLTAwMDAwMDAwMDAwMCIsImV4cCI6MTcyMjk2NDU2MSwicm9sZXMiOlsiRURJVE9SIiwiV09SS1NQQUNFX0VESVRPUiIsIldPUktTUEFDRV9SRUFERVIiLCJPUkdBTklaQVRJT05fRURJVE9SIiwiQVVUSEVOVElDQVRFRF9VU0VSIiwiT1JHQU5JWkFUSU9OX01FTUJFUiIsIldPUktTUEFDRV9BRE1JTiIsIk9SR0FOSVpBVElPTl9SRUFERVIiLCJSRUFERVIiLCJBRE1JTiIsIk9SR0FOSVpBVElPTl9BRE1JTiJdfQ.VF3GlYLQHmAGxY1kjZqprtreY6ayhfoxEN3QuJdib8E''',
   (...)
     20 #     ),
     21 # )
     24 s = airbyte_api.AirbyteAPI(
     25     server_url="<http://localhost:8000>",
     26     security=models.Security(
   (...)
     32     ),
     33 )
---&gt; 35 res = s.connections.get_connection(request=api.GetConnectionRequest(
     36     connection_id='cdd4bf7e-ed72-406a-a882-6d9a1f6adcd1',
     37 ))

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/connections.py:149, in Connections.get_connection(self, request)
    147     _, e = self.sdk_configuration.get_hooks().after_error(AfterErrorContext(hook_ctx), None, e)
    148     if e is not None:
--&gt; 149         raise e
    151 if utils.match_status_codes(['403','404','4XX','5XX'], http_res.status_code):
    152     result, e = self.sdk_configuration.get_hooks().after_error(AfterErrorContext(hook_ctx), http_res, None)

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/connections.py:144, in Connections.get_connection(self, request)
    142 try:
    143     req = client.prepare_request(requests_http.Request('GET', url, params=query_params, headers=headers))
--&gt; 144     req = self.sdk_configuration.get_hooks().before_request(BeforeRequestContext(hook_ctx), req)
    145     http_res = client.send(req)
    146 except Exception as e:

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/_hooks/sdkhooks.py:41, in SDKHooks.before_request(self, hook_ctx, request)
     39 def before_request(self, hook_ctx: BeforeRequestContext, request: requests.PreparedRequest) -&gt; requests.PreparedRequest:
     40     for hook in self.before_request_hooks:
---&gt; 41         out = hook.before_request(hook_ctx, request)
     42         if isinstance(out, Exception):
     43             raise out

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/_hooks/clientcredentials.py:59, in ClientCredentialsHook.before_request(self, hook_ctx, request)
     55 session_key = self.get_session_key(
     56     credentials.client_id, credentials.client_secret)
     58 if session_key not in self.sessions or not self.has_required_scopes(self.sessions[session_key].scopes, hook_ctx.oauth2_scopes) or self.has_token_expired(self.sessions[session_key].expires_at):
---&gt; 59     sess = self.do_token_request(credentials, self.get_scopes(
     60         hook_ctx.oauth2_scopes, self.sessions.get(session_key)))
     62     self.sessions[session_key] = sess
     64 request.headers["Authorization"] = f"Bearer {self.sessions[session_key].token}"

File ~/PycharmProjects/ai-insights/src/venv/lib/python3.11/site-packages/airbyte_api/_hooks/clientcredentials.py:122, in ClientCredentialsHook.do_token_request(self, credentials, scopes)
    119 response = <http://self.client.post|self.client.post>(token_url, data=payload)
    121 if response.status_code &lt; 200 or response.status_code &gt;= 300:
--&gt; 122     raise Exception(
    123         f"Unexpected status code {response.status_code} from token endpoint")
    125 response_data = response.json()
    127 if response_data.get("token_type") != "Bearer":

Exception: Unexpected status code 405 from token endpoint```
9

Can you try localhost:8001?

10

https://reference.airbyte.com/reference/standalone-server-deprecation-and-migration-to-airbyte-server

11

no connection there (8001)

12

geeeh I’ll try later in my local setup

13

thanks!
all looks good and working… maybe you have some permissions? coz request seemed to be authorized with credentials I provide (since when I change to wrong creds error is unauthorized) do you have some kind of permissions here for api usage?

14

<@U01MMSDJGC9> any chance you were able to check this?
I tried to dig deeper with no results
even tried other OS with different laptops. all the same authorized but forbidden.

15

<@U01MMSDJGC9> looks like it’s not working for many new users with abctl latest version

16 
    server_url="<http://localhost:8000>",
    security=models.Security(
        client_credentials=models.SchemeClientCredentials(
            client_id=client_id,
            client_secret=client_secret,
            TOKEN_URL="v1/applications/token",
        ),
    ),
)
res = s.health.get_health_check()

if res is not None:
    print(res.raw_response)```
17

This work for my local abctl install.

18

<@U01MMSDJGC9> yep this code works (200)

19

but then I still cant make any api calls
405 or forbidden error remains

20

I also believe thats not about python sdk
since same problem when I call API directly