Normalization error clickhouse

Clickhouse normalization error. This error is from the day clickhouse destination was released

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu 20.04
  • Memory / Disk: 64GB, 8TB
  • Deployment: Docker compose
  • Airbyte Version: 0.40.4
  • Source name/version: Postgres/1.0.7
  • Destination name/version: Clickhouse/0.1.12
  • Step: Normalization
  • Description:

Database Error in model test_my_dbcategory (models/generated/airbyte_incremental/public/test_my_dbcategory.sql)

DB::Exception: Missing columns: ‘true’ while processing query: ‘WITH __dbt__cte__test_my_table_ab1 AS (SELECT JSONExtractRaw(assumeNotNull(_airbyte_data), ‘id’) AS id, JSONExtractRaw(assumeNotNull(_airbyte_data), ‘category’) AS category, _airbyte_ab_id, _airbyte_emitted_at, now() AS _airbyte_normalized_at FROM public._airbyte_raw_test_my_table AS table_alias WHERE 1 = 1), __dbt__cte__test_my_table_ab2 AS (SELECT accurateCastOrNull(id, ‘\n Float64\n’) AS id, nullIf(accurateCastOrNull(replaceRegexpAll(category, concat(’[1]|[‘, regexpQuoteMeta(’"'), ']$‘), ‘’), ‘String’), ‘null’) AS category, _airbyte_ab_id, _airbyte_emitted_at, now() AS _airbyte_normalized_at FROM __dbt__cte__test_my_table_ab1 WHERE 1 = 1), __dbt__cte__test_my_table_ab3 AS (SELECT assumeNotNull(hex(MD5(concat(toString(id), ‘~’, toString(category))))) AS _airbyte_test_offerW__pp_appcategory_hashid, tmp.* FROM __dbt__cte__test_my_table_ab2 AS tmp WHERE 1 = 1) SELECT id, category, _airbyte_ab_id, _airbyte_emitted_at, now() AS _airbyte_normalized_at, _airbyte_test_offerW__pp_appcategory_hashid FROM __dbt__cte__test_my_table_ab3 WHERE (1 = 1) AND coalesce(CAST(_airbyte_emitted_at, ‘DateTime64’) >= (identity(_CAST(‘0’, ‘Nullable(DateTime64(3))’)) AS _subquery3724), true)’, required columns: ‘_airbyte_ab_id’ ‘category’ ‘_airbyte_test_offerW__pp_appcategory_hashid’ ‘_airbyte_emitted_at’ ‘id’ ‘true’ ‘_airbyte_ab_id’ ‘category’ ‘_airbyte_test_offerW__pp_appcategory_hashid’ ‘_airbyte_emitted_at’ ‘id’ ‘true’.


  1. ‘, regexpQuoteMeta(’"'), ’ ↩︎

Hey could you share the complete log file for this sync?

Hi I sent it to you on slack, There are some tables and fields I don’t think I can post here.

Hey replied on slack