Confusion about WASS implementation and ClickHouse replacingmergetree

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

User is confused about how the WASS implementation and ClickHouse replacingmergetree work together, specifically regarding the potential issue of WAL logs causing rows to be stored in the wrong order and dedupe incorrectly.


Question

Hi all, I was reading the recent blog about new WASS implementation here
https://airbyte.com/blog/supporting-very-large-cdc-syncs-with-wass

So I airbyte-connectors to push into ClickHouse in a replacingmergetree but seeing this have me confused that if WAL logs changes will be thrown in-between my replacingmergetree will store rows in wrong order and will dedupe incorrectly. Please help me if I’ve understood it wrong.


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["wass-implementation", "clickhouse-replacingmergetree", "wal-logs", "dedupe"]