Airbyte lookup values

Hi

Completely new to AIrbyte. I have a question…let’s say I want Airbyte to pull data from system-xxx. Now, I don’t want to pull 100% of data from system-xxx; rather, I want to be able to pull the data from system-xxx with the values of ‘123’ and ‘456’ in one particular index field.

Assume further that ‘123’ and ‘456’ should not be hard coded in the Airbyte configuration, but these values could change. They are stored in a Snowflake database. Is there a way I can get Airbyte to go to Snowflake to get these “lookup values” and then use them to pull data from system-xxx?

Hey @Je Sum,
Airbyte sticks to ELT philosophy and focuses on the Extract and Load steps to retrieve raw data from a source and send it to a warehouse destination.
Our source database connector does not enable user to build predicates to filter source data.
A workaround I could suggest is to build a view on your source that represents your filtered table and sync this view with Airbyte.

1 Like