Feature Store Evaluation

What is a Feature Store?

Feature stores have become a critical component of the modern ML stack. They automate and centrally manage the data processes that power operational ML models in production, and allow data practitioners to build and deploy features quickly and reliably. Read more about what a feature store is and check out the additional resources below.

Feature Store Comparison

The MLOps Community has worked with vendors and community members to profile the major solutions available in the market today, based on our feature store evaluation framework.

Sort by
    • History:

      Co-created by GO-JEK and Google Cloud, now governed by the Linux Foundation with Tecton as main contributor

    • Stand-alone vs. Platform:

      Stand-alone feature store, integrates with 3rd party MLOps platforms

    • Delivery Model:

      Open source

    • Clouds Supported:

      AWS, GCP, Azure, On-Prem

    • Service Level Guarantees:

      None

    • Support:

      N/A (open source only)

    • History:

      Founded by the creators of Uber's Michelangelo platform

    • Stand-alone vs. Platform:

      Stand-alone feature store, integrates with 3rd party MLOps platforms

    • Delivery Model:

      Fully-managed cloud service

    • Clouds Supported:

      AWS (now), GCP and Azure (roadmap)

    • Service Level Guarantees:

      Uptime, Serving latencies

    • Support:

      24 x 7 support & response time guarantees

    • History:

      Developed internally by AWS

    • Stand-alone vs. Platform:

      Part of the Amazon SageMaker platform

    • Delivery Model:

      Fully-managed cloud service

    • Clouds Supported:

      AWS

    • Service Level Guarantees:

      None

    • Support:

      24 x 7 support & response time guarantees

  • short demo

    Video Coming Soon

    • History:

      Created by Databricks

    • Stand-alone vs. Platform:

      Part of a broader ML Ops platform

    • Delivery Model:

      Fully-Managed Cloud Service

    • Clouds Supported:

      AWS, GCP, Azure

    • Service Level Guarantees:

      Uptime

    • Support:

      24 x 7 support & response time guarantees

  • short demo

    Video Coming Soon

    • History:

      First developed at KTH University, now managed by startup Logical Clocks

    • Stand-alone vs. Platform:

      Part of the Hopsworks MLOps platform

    • Delivery Model:

      Open source, self-managed commercial, and fully-managed cloud service

    • Clouds Supported:

      AWS and Azure (managed service), GCP and on-prem (self-managed)

    • Service Level Guarantees:

      Uptime, Serving latencies

    • Support:

      24 x 7 support & response time guarantees

    • History:

      Feature store created by Iguazio. Includes open source components created and maintained by Iguazio

    • Stand-alone vs. Platform:

      Part of the Iguazio Data Science Platform

    • Delivery Model:

      Open source components, self-managed commercial, and fully-managed cloud service

    • Clouds Supported:

      AWS, GCP, Azure, On-Prem

    • Service Level Guarantees:

      Uptime

    • Support:

      24 x 7 support & response time guarantees

  • short demo

    Video Coming Soon

    • History:

      Founded by Patrick Dougherty and Jared Parker

    • Stand-alone vs. Platform:

      Stand Alone Feature Store

    • Delivery Model:

      Open Source Software

      Fully-Managed Cloud Service

    • Clouds Supported:

      AWS, GCP, Azure, On-Prem

    • Service Level Guarantees:

      Uptime

    • Support:

      24 x 7 support & response time guarantees

    • History:

      Originally created by Venkata Pingali and Indrayudh Ghoshal (founders)

    • Stand-alone vs. Platform:

      Stand-alone feature store

    • Delivery Model:

      Self Managed Commercial or Fully Managed Cloud service

    • Clouds Supported:

      On AWS, GCP, On-Prem

    • Service Level Guarantees:

      Uptime

    • Support:

      24 x 7 support & response time guarantees

How to choose a feature store

Are you looking to add a feature store to your ML stack? MLOps Community, with the help of feature store vendors, has created an evaluation framework to help you choose the right product for your needs.

Criteria 1

Commercial Information

First, you need to assess whether the product’s commercial characteristics meet your needs. We recommend evaluating the following commercial criteria:

  • Delivery Model: Open source or managed service? 
  • Standalone feature store or part of a broader ML platform? 
  • Is the product available on-premises and / or in your public cloud?
  • Is the product delivered as commercial software, open source software, or a managed cloud service?
  • What is the pricing model?  
  • SLOs / SLAs: Does the vendor provide guarantees around service levels?
  • Support: Does the vendor provide 24×7 support?

Criteria 2

Feature store capabilities

You will want to make sure that the feature store fulfills all the capabilities you need across the operational data workflow. We’ve broken down the capabilities as follows:

Feature Definitions
Does the feature store provide a framework for creating feature definitions (including the transformation logic and materialization), and can data scientists collaborate on the definitions?

Automated Transforms:
Does the feature store automatically execute the pipelines required to process the feature values, including historical backfill and fresh feature values? Do the transformations support batch, streaming and real-time data sources?

Feature Ingestion:
How are features ingested into the online and offline store?

Storage and Feature Processing Infrastructure:
What infrastructure does the feature store use to store and process feature values?

Feature Sharing & Discovery:
Is there an easy way to manage, share and discover features across the organization?

Online Serving:
How are features served online at inference time?

Training Datasets:
How do data scientists generate point-in-time accurate training datasets from the offline store?

Monitoring and Alerting:
What monitoring and alerting capabilities does the feature store provide?

Security and Data Governance:
What measures are in place to protect data and control access?

Integrations:
Which 3rd party data and ML tools does the feature store integrate with?