Boxkite simplifies model monitoring by capturing feature and inference distributions used in model training and comparing them against real time production distributions via Prometheus and Grafana
Free, open source
0.5 seconds to process 1 million data points (training)
Sub millisecond p99 latency (serving)
Supports sampling for large data sets
Aggregates histograms from multiple server replicas (using PromQL)
Separate counters for discrete and continuous variables (ie. categorical and numeric features)
Initializes serving histogram bins from training data set (based on Freedman-Diaconis rule)
Handles unseen data, nan, None, inf, and negative values
One metric for each counter type (no confusion over which metric to choose)
Default configuration supports both feature and inference monitoring (easy to setup)
Small set of dependencies: prometheus, numpy, and fluentd
There are no reviews yet. Be the first to write one.
Vendor |
Demo link |
Summary |
How much does it cost? |
Video/Tutorials |
What’s a sample use case? Where can I learn from? |
Feature List |