Optimizing CDC processing for Airbyte

Summary

Exploring options to optimize CDC processing in Airbyte for parallel thread execution


Question

Hello, I understand that CDC through airbyte currently processes up to 10,000 rows at a time in the created job. Is there a part that can be tuned and divided into parallel threads to process up to 10,000 or more? This is to shorten the time when backfill.



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["cdc-processing", "airbyte-platform", "parallel-threads", "backfill"]