Summary
The user is looking to manually adjust the number of buckets in tables created automatically by Airbyte for EL processes into a Doris database.
Question
Hi everyone,
I’m encountering an issue and would like to ask for some help:
I’m using Airbyte for EL (Extract and Load) processes to load data into a Doris database. During this process, Airbyte automatically creates tables with the following SQL:
The issue I’m facing is that the default number of buckets is set to 16, but I would like to manually adjust this number, for example, to 8.
Is there a way to manually set or change the number of buckets in the tables created automatically by Airbyte?
Thanks for your help!
CREATE TABLE IF NOT EXISTS `lineitem` ( `_airbyte_ab_id` varchar(40), `_airbyte_emitted_at` BIGINT, `_airbyte_data` String ) DUPLICATE KEY(`_airbyte_ab_id`, `_airbyte_emitted_at`) DISTRIBUTED BY HASH(`_airbyte_ab_id`) BUCKETS 16 PROPERTIES ( "replication_allocation" = "tag.location.default: 1" );
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.