Hey guys,
We have worked with Fivetran & Dataddo, looking for an alternative and AirByte poped-up.
From the web/docs it’s not clear to me that it’s possible to customize the data ingestion from the sources.
In Dataddo we can ingest with any SQL select query, useful for doing aggregations and filtering data from the start.
Can this be done with AirByte?
+1
@Community, is there any way at all to ingest partial data from a source? E.g. even a subset of columns to start with. The holy grail, of course, would be “ingest with any SQL select query”.
If not possible today, is there a way one could code this on top of Airbyte without too significant an engineering effort?
H!
Ingestion of subset of columns is possible as of release 0.50 - see this article for more details: Announcing Airbyte 0.50: Checkpointing, Column Selection, and Schema Propagation | Airbyte
It’s not possible though to use any filter predicates or custom SQL queries. No idea if something like that is in the roadmap/backlog (I’m just a user of Airbyte, not employee).
BR,
Kuba