I’m developing a connector for an ERP system with multiple streams. When there is a stream sync failure, is it possible to continue at the stream that failed during the 2nd sync attempt instead of starting all streams over again? Any input appreciated.
Yes if the stream comes from a Source API https://docs.airbyte.com/connector-development/cdk-python/incremental-stream#interval-based-checkpointing
Thank you for the quick reply. Is there any way to continue a sync operation consisting of multiple streams at the stream which had the failure (a connection timeout for example) instead of syncing all streams again. Example:
Begin sync containing streams A, B, and C. Stream A completes syncing. Stream B completes syncing. Stream C has a sync failure. Airbyte begins Attempt 2 and starts with Stream A again.
In the scenario above, is it possible to just start with stream C, as A and B already completed?
Thank you again.
This is a feature in progress. Probably will release in early July.
Is there anywhere to track this feature so I know when it is available? Thanks.
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.