Weights & Biases

Model Store Overview

Vendor Name

Weights & Biases

Stand-alone vs. Platform

Standalone platform that is infrastructure-agnostic.

Delivery Model

Client library is open source, and the server and UI interface is closed source.

Clouds Supported

GCP, AWS, Azure an On-prem

Pricing Model

Free for individuals and academics.
Usage based pricing for commercial enterprises.

Service Level Guarantees

Service level guarantees are plan-dependent.

Support

Support is available during regular business hours online, via Slack, and over the phone for all customers.
Additional support is provided for all enterprise customers.

SSO, ACL

SAML, OIDC, Active Directory, PingFederate or OAuth2 identity providers.
Access controls can be asserted at the team and project levels.

Security and Compliance

GDPR and SOC 2 Type 2 compliant.

Model Store Capabalities

Setup

You can get started with W&B in three lines of code on their cloud hosted infrastructure.
When self hosting W&B, you can run a trial instance on any machine, with Docker in 5 minutes.
Production deployments generally take less than 1 hour to provision.

Flexibility, Speed, and Accessibility

W&Bs API is designed to be flexible, allowing arbitrarily flexible structures.
Framework, infrastructure, model, and data agnostic.
Designed to be scalable for the needs of the enterprises by handling 10s of millions of models trained concurrently.

Model Versioning, Lineage, and Packaging

Captures the entire lineage of all data and code that was executed to produce a given model.

Log and Display of Metadata

You can log as much metadata and config data as you like. Metadata is logged in the runs table to enable comparison of this data across runs and is also logged in the ""summary"" section of each individual run.
Code saving turns on all saving of the notebook or the script in which W&B was run. The current git commit is also stored where there is a git file present.
All kinds of metrics data can be logged and monitored live. Custom charts can also be used for advanced charting capability.
System metrics such as CPU and GPU usage and memory allocation are logged. Gradients from the model can be logged and visualised
Image, text, audio, video, molecule, html data can all be logged and visualised in the W&B workspace for a particular run as well. "

Comparing Experiments and Models

You can compare experiments with:
Overlayed loss curves, learning curves, metrics curves + table with all metrics
Parallel coordinates plot
Run comparer to easily diff between run configs
Diff viewer to diff between saved code
Tables viewer to compare predictions from multiple models on the same dataset"

Organizing and Searching Experiments and Models

Find experiments by name, tag or group, including using wildcards
Filter experiments by config parameters or results
Organise all experiments within Projects
Customise the workspace to an individual users' preference to enable easy access to the most relevant information for each user"

Model Review, Collaboration, and Sharing

One of W&B's standard tools are Reports, a collaborative word editor that enables reporting and sharing of results.
All panels logged in the W&B Workspace can be imported into a Report and remain fully interactive. They update in real time in the Report.
Reports enable embedding of image, video, audio, social media links and have a rich array of word editing formatting options (like LaTeX) as well as the ability to add comments to different charts or Report sections
Reports can be shared publically, or just privately among a W&B Team

CI/CD/CT Compatibility

Programmatic APIs to all data stored in its system of record.
Users trigger arbitrary actions in their CI or CD systems using these APIs.

Integrations

Designed to integrate easily with all popular ML libraries.

Reviews

There are no reviews yet. Be the first to write one.