Hello,
My team is using Airbyte to manage thousand of jobs every day. After those jobs are done, other processes are launched. To know if the jobs are finished we used the Airbyte API like (the Airflow sync operator is doing). However, we found it counter productive to make many call to monitor thousand of jobs every day. Is the Airbyte team planned to implement some sort of callback ?
I know there is a webhook that can be used (for slack for example). However, we are not sure if it is suitable to use it to trigger following processes at scale.