Trouble using Airflow <> Airbyte connector version 4.0.0

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

User is facing authentication errors when trying to refer to a connection in their Airflow DAG using the Airflow <> Airbyte connector version 4.0.0.


Question

Hello :wave:
Has anyone managed to use <https://airflow.apache.org/docs/apache-airflow-providers-airbyte/stable/index.html|Airflow <> Airbyte connector> (version 4.0.0)?
Despite my connection passed the connection_testin Airflow, I can’t refer to it in my dag. I keep having auth errors (cf thread).


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["airflow", "airbyte-connector", "authentication-errors", "connection-test", "dag"]
2 
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/airbyte/hooks/airbyte.py", line 157, in submit_sync_connection
    res = self.airbyte_api.jobs.create_job(
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/jobs.py", line 100, in create_job
    raise e
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/jobs.py", line 95, in create_job
    req = self.sdk_configuration.get_hooks().before_request(BeforeRequestContext(hook_ctx), req)
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/_hooks/sdkhooks.py", line 41, in before_request
    out = hook.before_request(hook_ctx, request)
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/_hooks/clientcredentials.py", line 59, in before_request
    sess = self.do_token_request(credentials, self.get_scopes(
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/_hooks/clientcredentials.py", line 122, in do_token_request
    raise Exception(
Exception: Unexpected status code 404 from token endpoint
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 767, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 733, in _execute_callable
    return ExecutionCallableRunner(
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/operator_helpers.py", line 252, in run
    return self.func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/baseoperator.py", line 406, in wrapper
    return func(self, *args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/airbyte/operators/airbyte.py", line 83, in execute
    job_object = hook.submit_sync_connection(connection_id=self.connection_id)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/airbyte/hooks/airbyte.py", line 165, in submit_sync_connection
    raise AirflowException(e)
airflow.exceptions.AirflowException: Unexpected status code 404 from token endpoint
[2024-10-30, 16:20:37 UTC] {taskinstance.py:1225} INFO - Marking task as FAILED. dag_id=trigger_airbyte_sync, task_id=trigger_airbyte_sync_task, run_id=manual__2024-10-30T16:20:35.192103+00:00, execution_date=20241030T162035, start_date=20241030T162037, end_date=20241030T162037
[2024-10-30, 16:20:37 UTC] {taskinstance.py:340} â–¼ Post task execution logs
[2024-10-30, 16:20:37 UTC] {standard_task_runner.py:124} ERROR - Failed to execute job 86 for task trigger_airbyte_sync_task (Unexpected status code 404 from token endpoint; 10957)
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/airbyte/hooks/airbyte.py", line 157, in submit_sync_connection
    res = self.airbyte_api.jobs.create_job(
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/jobs.py", line 100, in create_job
    raise e
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/jobs.py", line 95, in create_job
    req = self.sdk_configuration.get_hooks().before_request(BeforeRequestContext(hook_ctx), req)
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/_hooks/sdkhooks.py", line 41, in before_request
    out = hook.before_request(hook_ctx, request)
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/_hooks/clientcredentials.py", line 59, in before_request
    sess = self.do_token_request(credentials, self.get_scopes(
  File "/home/airflow/.local/lib/python3.10/site-packages/airbyte_api/_hooks/clientcredentials.py", line 122, in do_token_request
    raise Exception(
Exception: Unexpected status code 404 from token endpoint
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/task/task_runner/standard_task_runner.py", line 117, in _start_by_fork
    ret = args.func(args, dag=self.dag)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/cli_config.py", line 49, in command
    return func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/cli.py", line 115, in wrapper
    return f(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/task_command.py", line 483, in task_run
    task_return_code = _run_task_by_selected_method(args, _dag, ti)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/task_command.py", line 256, in _run_task_by_selected_method
    return _run_raw_task(args, ti)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/task_command.py", line 341, in _run_raw_task
    return ti._run_raw_task(
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/session.py", line 97, in wrapper
    return func(*args, session=session, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 3004, in _run_raw_task
    return _run_raw_task(
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 273, in _run_raw_task
    TaskInstance._execute_task_with_callbacks(
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 3158, in _execute_task_with_callbacks
    result = self._execute_task(context, task_orig)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 3182, in _execute_task
    return _execute_task(self, context, task_orig)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 767, in _execute_task
    result = _execute_callable(context=context, **execute_callable_kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 733, in _execute_callable
    return ExecutionCallableRunner(
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/operator_helpers.py", line 252, in run
    return self.func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/baseoperator.py", line 406, in wrapper
    return func(self, *args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/airbyte/operators/airbyte.py", line 83, in execute
    job_object = hook.submit_sync_connection(connection_id=self.connection_id)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/airbyte/hooks/airbyte.py", line 165, in submit_sync_connection
    raise AirflowException(e)
airflow.exceptions.AirflowException: Unexpected status code 404 from token endpoint
[2024-10-30, 16:20:37 UTC] {local_task_job_runner.py:266} INFO - Task exited with return code 1
[2024-10-30, 16:20:37 UTC] {taskinstance.py:3900} INFO - 0 downstream tasks scheduled from follow-on schedule check```
3

I think this might be related to this <Issues · apache/airflow · GitHub issue>.