How does the flow of airbyte goes in incremental dedup history mode

Q&A
1

hi,
I just want to know how extra tables in the airbyte is used. As per my analysis from doc i have found following flow:

step-1. destination connector provides the last sync state to source connector
step-2. source connector compares and sends only updated data
step-3. destination receives it and append to raw table (which is never deleted)
step-4. Later on raw table data is converted to destination schema scd table (normalization incrementally based)
step-5. scd table data is pulled to stg table (stg table contains only new rows in dest. schema)
step-6. stg data is pushed to final table (update based on 3 columns - cdc_deleted_at, cdc_updated_at)

But in logs ,i found that data is added in following sequences --1) stg table 2) scd table and 3) final table
can anyone confirm how the flow goes?

2

Historical context @Yash09 a few versions ago Airbyte didn’t support incremental update in normalization. This causes that every time Normalization recreate the SCD and Final table. After implementing this incremental update normalization will generate the STG table where stores only the new records.
If you want to see the complete model you can generate the dbt docs which create a lineage for your data. Check how to extract normalization models of the Airbyte.