Fennel - Next Generation Data Pipelines

Fennel Next Generation Data Pipelines

Contact for Pricing

#data pipelines
#Machine Learning
#Real Time
#batch processing
#feature store
#data quality

What is Fennel?

Fennel is a platform designed to help users author, compute, store, serve, monitor, and govern both real-time and batch machine learning pipelines. It utilizes a CDC-aware engine built in Rust for automatic incremental computation, optimizing efficiency and ensuring data freshness. The system avoids DSLs, relying instead on standard Python and Pandas, thereby minimizing the learning curve.

Features include automatic backfills, fully-managed infrastructure, and a feature repository for reusability. Fennel also incorporates robust data quality tooling, including strong typing, immutability, versioning, unit testing, compile-time validation, and data expectation checks to prevent errors and maintain data integrity.

Features

Incremental Computation: Automatically computes only the changes in data, improving efficiency.
Python-Based: Uses standard Python and Pandas, eliminating the need for DSLs or specialized job configurations.
Automatic Backfills: Pipelines automatically backfill on declaration.
Fully-Managed Infrastructure: Manages all necessary infrastructure components.
Feature Repository: Allows users to write standardized features once and reuse them across different applications.
Strong Typing: Catches typing bugs at compile time and at the source of data.
Immutability & Versioning: Features are immutable and versioned to avoid discrepancies between offline and online environments.
Unit Testing: Supports unit testing for both batch and real-time pipelines.
Compile Time Validation: Validates lineage end-to-end at compile time to minimize runtime errors.
Data Expectation: Allows users to define expected data distributions.
Online / Offline Skew: single definition of feature.