Deepchecks

What do you do?

Deepchecks is a minimally intrusive MLOps solution for continuous validation of machine learning systems, meant to enable you to trust your models through the continuous changes in your data lifecycle. Deepchecks includes the must-have features for any ML Monitoring system: performance monitoring, data drift detection and anomaly detection alerts, along with some unique features that are especially helpful for complex ML pipelines: Monitoring various phases of the pipeline, detecting hidden data integrity issues, detecting low confidence segments, detecting inconsistencies that are hidden within unstructured text, etc.

Deepchecks can fit into existing pipelines within all of the major cloud platforms, as well as some on-prem/hybrid architectures.

How much does it cost?

Free trial and then ongoing costs are based on the number of models & the amount of analyzed data

What’s a sample use case? Where can I learn from?

To get a feel of our system, please try out our sandbox. It’s worth noting that this is a version with limited screens and functionality, the full version can be opened for experimentation upon request.

Examples of use cases can be found on our website:

We also support use cases such as AI based cyber-security systems, underwriting, asset evaluation, ad-tech (bidding, personalization, etc.), e-commerce (conversion from cart, recommender systems)

Feature List

  • Feature-rich alerting system: Out-of-the-box configured alerts constantly running QA & notifying you about schema mismatches, data integrity issues, algorithmic bias, anomalies, segments in which a challenger model is better, etc.
  • Observability of ML in production: Customizable dashboard that enables you to view and explore various real-time metrics related to your ML systems while comparing between different segments, time frames, etc.
  • Multi-phase monitoring: Deepchecks taps into your ML system at various points of interest – training vs. production, before and after preprocessing, after applying business logic, in multiple models during A/B tests, and utilizes this to notify about issues that would otherwise go unnoticed
  • Versatile operation modes: Support of both stream and batch processes, and support of partial, biased or no ground truth labels coming back from production.
  • Detecting mismatches between training and production environments: Since Deepchecks can connect both to the training data and the production data, it can notify you about mismatches related to the data scheme, distributions, etc. This includes cases in which the training and production have different distributions due to resampling.
  • Quick querying of problematic production data: Deepchecks enables you to always be one click away from running code on the relevant production data
  • Rich integration options: Integration can be done either via an API, or by granting Deepchecks read access to data storage such as S3, GCS, BigQuery, etc. Support of both SaaS & on-prem architectures
  • Customizability: Ability to combine with custom code, create custom alerts, metrics and pipeline structures, while still maintaining a simple and intuitive user experience.
  • Intuitive UI: Our users love it! Try our online sandbox to “feel” the limited version.