Summary
User encounters a ValueConflictKnownException indicating a sync is already running for a specific connection_id while attempting to fetch data from multiple GitHub repositories and branches. Additionally, syncs are running continuously despite not being scheduled, leading to confusion and errors.
Question
Hi, I have created a pipeline from Github to Snowflake and orchestrated it using airflow. I’m reading from 1 repository, 2 branches and fetching data for 13 streams from each branch.
I’m getting following conflict error.
io.airbyte.commons.server.errors.ValueConflictKnownException: A sync is already running for: {connection_id}```
Also, syncs are continuously running even when they are not scheduled for the time. Syncs are getting succeeded but still getting conflict error.
My intend is to fetch data from 10 repos, 2 branch each and for 13 stream. Does any one have experience setting such pipeline? Any help is much appreciated.
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1718981130836709) if you want
to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
['github', 'snowflake', 'airflow', 'sync-error', 'valueconflictexception']
</sub>