Is this your first time deploying Airbyte: No
OS Version / Instance: Debian 10 (buster) on AWS EC2, 8 cores, 16GB RAM
Deployment: Docker
Airbyte Version: 0.35.45-alpha
Hi, I am using Airflow/Composer to trigger around 15 Airbyte jobs using differents DAG that all use the AirbyteTriggerSyncOperator. Among all of these DAGs, one in particular has a strange behavior ;
Sometimes, Airbyte would not communicate the result of the job, leaving the Airflow task in a “running” state, waiting for a signal from Airbyte (although the Airbyte job has run successfully) until it reaches its timeout…
The Airbyte logs does not provide any sucessful insight as the job obviously ran successfully.
Did you ever run in this problem before ? What’s steps can I do to reproduce/detect the problem ?
Hey @NahidOulmi,
I would help if you could share the Airflow logs and the Airbyte sync logs. When you encounter this problem it would be interesting to call Airbyte’s Job API manually to check the status of the job and identify if this behavior is a problem on Airbyte or Airflow operator side.
1 Like
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.