Determined AI

Model Store Overview

Vendor Name

Determined AI

Stand-alone vs. Platform

Stand-alone

Delivery Model

Open-source with a paid Enterprise edition that contains additional features and commercial support.

Clouds Supported

On-prem and self-hosted private cloud.

Pricing Model

GPU-based pricing.

Service Level Guarantees

Open-source: best effort.
Enterprise: Support SLAs guaranteed.

Support

For Enterprise.

SSO, ACL

Enterprise: yes. Full integration with SAML/SCIM.

Security and Compliance

Open-source: basic user management system.
Enterprise: RBAC.

Model Store Capabalities

Setup

Determined CLI can be pip-installed. A Determined cloud or on-premise environment can be provisioned via one "det deploy" command.

Arbitrary workloads can be run on Determined, but getting features like distributed training, HPO, and experiment tracking requires making model code adhere to Determined APIs that look similar to PyTorch lightning (but support TensorFlow, Keras, PyTorch).

Flexibility, Speed, and Accessibility

Determined provides a metadata store that is structured in a relational database which tracks experiment metadata, and exposed to users via a WebUI, Rest API, and Python API. Metadata includes: inputs (code, config, data, hyperparameters), intermediate data (training/validation metrics, logs, checkpoints, optimizer state), and outputs (versioned, trained model weights).

Model Versioning, Lineage, and Packaging

Supports:
* Model versioning via Model Registry
* Code versioning
* Metrics tracking

Log and Display of Metadata

-Experiment information
-Data versions are in experiment configuration
-Parameters/metrics
-Optimizer state
-Training logs

Comparing Experiments and Models

Extensive visualizations provided in webUI
Single trial: loss/metric curves, hyperparameters, workloads, optional profiling, indexed/searchable logs.
Experiments/HP Search: comparison tables, parallel loss/metric curves, configuration, HP parallel coordinate plots, HP heat maps, HP scatter plots.

All of the above can be compared across experiments as well.

Also offers full integration with TensorBoard.

Organizing and Searching Experiments and Models

Experiment organization is built into the tool and a natural consequence of adhering to the Determined APIs. Post-hoc organization via labels is supported, as is search.

Model Review, Collaboration, and Sharing

Models can be versioned/named and locked, and all experiments/models are tied to a user.

CI/CD/CT Compatibility

Determined jobs can be kicked off as part of a CI/CD pipeline, and the programmatic REST and Python APIs (as well as the CLI) provide a natural way to pull data into a CI/CD system.

Integrations

Integrations are provided for: Delta/Spark, DVC, Pachyderm for upstream data processing. Seldon/Algorithmia downstream.
These provide a blueprint for integrating with other tools.

Reviews

There are no reviews yet. Be the first to write one.