Filtering Source Data for Ingestion

Summary

User inquires about the ability to filter source data during ingestion to avoid pulling entire tables, specifically to limit data to the last year due to large table sizes.


Question

Hi! I see normalization and transformation has been deprecated. Is it possible to filter a source so you don’t have to ingest the entire table into the target? I have some large tables I would prefer to only pull the last year or so of data and would like to avoid pulling anything earlier than that



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['filter-source-data', 'ingestion', 'normalization', 'transformation', 'large-tables']