Filtering Source Data in Airbyte

Summary

Airbyte user inquiring about filtering source data to avoid ingesting entire tables into the target


Question

Hi! I see normalization and transformation has been deprecated. Is it possible to filter a source so you don’t have to ingest the entire table into the target? I have some large tables I would prefer to only pull the last year or so of data and would like to avoid pulling anything earlier than that



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["filtering-source-data", "ingest", "large-tables", "data-filtering"]

looks like kapaai was able to answer for me https://airbytehq.slack.com/archives/C01AHCD885S/p1718294337298839?thread_ts=1718294134.995699&cid=C01AHCD885S