Databricks Feature Store

Databricks Feature Store

Created by Databricks

Part of a broader MLOps platform

Fully-Managed Cloud Service

AWS, GCP, Azure

Consumption Pricing

Uptime

24 x 7 support & response time guarantees

Declarative framework for defining features (incl. transformations and materialization)

Feature ingestion jobs managed in notebooks

Feature definitions are managed with Delta as a backing layer and a metadata managed service for schema enforcement.

Automated pipeline orchestration

Managed Batch

Streaming and Real-Time Transformations

Automated backfill of historical data

Pipeline visualization

Spark (for batch feature ingestion)

Pandas (for batch feature ingestion)

SQL (for batch ingestion)

Spark Streaming (for streaming feature ingestion)

Online currently can be pushed asynchronously to Aurora, RDS MySQL, Azure DB for MySQL, and Azure SQL DB. GCP online sync is still WIP.

Web UI

Searchable feature catalog with metadata

Feature discovery including transformations, data lineage, and values

Feature versioning and dependency management

Dataset generated from offline storage using Python SDK

Integrated with MLflow artifacts to automate retraining feature fetch when linked as a feature store definition

Allows sync for pushing feature data to online stores (RDS, Azure SQL, etc.)

Data quality monitoring

Data remains in end-user's cloud account

ACL, SSO, RBAC

Batch: Delta (defined in Hive metastore)

Streaming ingest through Spark from Kafka, Kinesis, EventHubs.