Weights & Biases

Weights & Biases

Standalone platform that is infrastructure-agnostic.

Client library is open source, and the server and UI interface is closed source.

GCP, AWS, Azure an On-prem

Free for individuals and academics.
Usage based pricing for commercial enterprises.

Service level guarantees are plan-dependent.

Support is available during regular business hours online, via Slack, and over the phone for all customers.
Additional support is provided for all enterprise customers.

SAML, OIDC, Active Directory, PingFederate or OAuth2 identity providers.
Access controls can be asserted at the team and project levels.

GDPR and SOC 2 Type 2 compliant.

You can get started with W&B in three lines of code on their cloud hosted infrastructure.
When self hosting W&B, you can run a trial instance on any machine, with Docker in 5 minutes.
Production deployments generally take less than 1 hour to provision.

W&Bs API is designed to be flexible, allowing arbitrarily flexible structures.
Framework, infrastructure, model, and data agnostic.
Designed to be scalable for the needs of the enterprises by handling 10s of millions of models trained concurrently.

Captures the entire lineage of all data and code that was executed to produce a given model.

You can log as much metadata and config data as you like. Metadata is logged in the runs table to enable comparison of this data across runs and is also logged in the ""summary"" section of each individual run.
Code saving turns on all saving of the notebook or the script in which W&B was run. The current git commit is also stored where there is a git file present.
All kinds of metrics data can be logged and monitored live. Custom charts can also be used for advanced charting capability.
System metrics such as CPU and GPU usage and memory allocation are logged. Gradients from the model can be logged and visualised
Image, text, audio, video, molecule, html data can all be logged and visualised in the W&B workspace for a particular run as well. "

You can compare experiments with:
Overlayed loss curves, learning curves, metrics curves + table with all metrics
Parallel coordinates plot
Run comparer to easily diff between run configs
Diff viewer to diff between saved code
Tables viewer to compare predictions from multiple models on the same dataset"

Find experiments by name, tag or group, including using wildcards
Filter experiments by config parameters or results
Organise all experiments within Projects
Customise the workspace to an individual users' preference to enable easy access to the most relevant information for each user"

One of W&B's standard tools are Reports, a collaborative word editor that enables reporting and sharing of results.
All panels logged in the W&B Workspace can be imported into a Report and remain fully interactive. They update in real time in the Report.
Reports enable embedding of image, video, audio, social media links and have a rich array of word editing formatting options (like LaTeX) as well as the ability to add comments to different charts or Report sections
Reports can be shared publically, or just privately among a W&B Team

Programmatic APIs to all data stored in its system of record.
Users trigger arbitrary actions in their CI or CD systems using these APIs.

Designed to integrate easily with all popular ML libraries.