ClearML

Model Store Overview

Vendor Name

ClearML

Stand-alone vs. Platform

Part of a broader platform

Delivery Model

Open-source with closed-source paid features

Clouds Supported

Ability to be deployed on-prem and all clouds either self hosted with OSS or managed with Enterprise

Pricing Model
Service Level Guarantees

Standard SLA that can be customized for Enterprise customers

Support

Open-source: community
Enterprise: customizable 24X7 SLA based on needs

SSO, ACL

ClearML Server supports self user management.
Enterprise offers full user management with SSO and user roles and permissions

Security and Compliance

SSO and LDAP in Enterprise as well as user management and roles and resource-based permissions

Model Store Capabalities

Setup

Server:
OSS: supports docker based installation, takes 10-30 minutes to set up as well as run on a Kubernetes Cluster Free, Pro & Enterprise: Fully Managed
SDK:
2 lines of code integrate to user's codebase. Same integration for all tiers.

Flexibility, Speed, and Accessibility

Any type of artifact can be saved (Files, folders and python objects), most artifacts have previews.
Multiple frameworks models are automatically captured.
Metadata can be accessed from python code or REST API.
Supports storage on various mediums (NFS, Azure blob, S3, etc...)

Model Versioning, Lineage, and Packaging

Model artifacts are stored alongside it's inputs (code, params, data, metrics).
Supports full reproducibility of models.
Supports tagging models and publishing them.
ClearML Agent also supports reproducing experiment including environment (almost 0 setup needed).

Log and Display of Metadata

Experiment information is automatically logged:
Git info, docker container info, parameters (using argparse, click & Hydra).
Model checkpoints from multiple frameworks such as pytorch, TF, sklearn are automatically logged.
Everything reported to Tensorboard or matplotlib is automatically captured.
Users can manually report scalars histograms, plots and custom plotly objects as well as images, audio files and console output.
Machine monitoring (GPU, CPU, Mem, Network)
Live tracking is supported.

Comparing Experiments and Models

Full experiment comparison including code, docker, parameters, output metrics.
Parallel coordinates for Hyperparameter optimization.

Organizing and Searching Experiments and Models

Organized into projects and sub-projects.
Further management with status and tags is available.
Has API for retrieving tasks according to filters.

Model Review, Collaboration, and Sharing

Models and experiments can be reviewed and tracked with built-in markdown editor.
Supports sharing views between colleagues.
Supports locking models for production.

CI/CD/CT Compatibility

Supports continuous integration with triggers and scheduling mechanisms.
Job run via ClearML Agent as the orchestrator.
Pipeline mechanism also available for complex processes.

Integrations

Integrates with a host of tools.
3rd party integrations (Slurm, Apache Airflow, and more) are also available.

Reviews

There are no reviews yet. Be the first to write one.