Cloning Airbyte open source repo with specific Clickhouse destination version

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

Cloning Airbyte open source repo into a VM with a specific Clickhouse destination version and data normalization concerns.


Question

Hi Team,

I want to clone Airbyte open source repo into a VM with Clickhouse destination version 0.2.5?

Is this doable?

The reason I ask is; current airbyte open source repo has Clickhouse destination version as 1.0 and it does not normalize data.


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["clone", "airbyte-open-source", "clickhouse-destination", "data-normalization"]
2

We’ve removed normalization support on most destinations and the platform itself — and we strongly advise that you do normalization on destination side with tools built for that purpose, dbt or anything you like.

You can hypothetically run an older release of airbyte that supports normalizations, and use destination clickhouse 0.2.5 as a custom connector (add its image as a custom connector to pin the version), but you won’t get updates and bug fixes that way.

You could try most recent airbyte platform with pinned custom destination, but I’m not sure that would work at all.