MLflow

MLflow

Stand-alone as an open-source project
Managed service as part of the Databricks platform

Available both as the open-source software and a managed solution as a part of the Databricks platform.

The open-source version is compatible on all clouds
Databricks platform can be deployed on Google Cloud, AWS, and Microsoft Azure

For the Databricks platform, it is usage-based (consumption) pricing

Open-source: None
Databricks platform: there are service levels guarantees that depend on your plan

Open-source: community
Databricks platform: you can get up to 24 x 7 support and response times guarantees that depend on your support plan

Open-source: None
Databricks platform: advanced user management options

Open-source: None
Databricks platform: enterprise level

Open-source:
For individual use or testing, you can have on a local filesystem.
For proper deployments, you need to set up and scale backend databases and maintain them.
You need to set up separate infrastructure and MLflow servers for every project.
Databricks platform: fully managed solution.

Open-source: Metadata structure is static. MLflow doesn't support nested parameter structures or custom log/artefact subfolders to organize model metadata.
Depending on the metadata and artefact backends, infrastructure setup, and the number of runs, experiments and models, the web UI can slow down drastically.
Databrick platform: For managed platform, you should have no problems with the speed of the UI.
Apart from MLflow querying API, you can use webhooks to access models.

Supports:
Packaging and versioning models in the MLflow model registry.
Visibility into model-related activity once the model is registered.
Doesn't support:
Full data lineage
Full experiment history and lineage from before it was registered

Supports:
Source control via git SHA
Execution info: start/end time + which file was an entry point
Parameters and metrics
Artifacts including image file formats, persisted models and data
Some things it doesn’t support:
Lists of metadata: for example, a list of prediction images with training step
Hardware metrics monitoring (CPU/Memory)
Rich formats such as audio/video

Comparison plots available:
Table with parameter/metric diff
Parallel coordinates plot
Overlayed learning curves
Parameter/metric scatter plot
Contour plot
Model schema (input-output)

Open-source:
Separate view for experiments and registered (production) models
Ability to save and display many versions for any registered model
Ability to search/filter by parameter/value with a query language
Ability to customize columns
Must organize runs into experiments (could also be limiting)
Not able to organize into workspaces and projects
Not capable of saving filtered table view for later
Tags management is not supported in the UI
Databrick platform:
Ability to use the workspace to organize models and experiments

Open-source: Ability to transition models between production stages (Staging, Production or Archived) via MLflow API.
Databrick platform: Ability to review, comment on, and approve models on the platform. Once approved, they can be moved between production stages. Databricks platform provides many collaboration features, including comments, webhooks and notifications.

Open-source: Ability to integrate inside CT workflows, but you need to implement it.
Databrick platform: Ability to trigger workflows based on webhooks and run tests based on production stage transitions.

Open-source: MLflow integrates with many libraries from the ML ecosystem, providing autologing and model packaging for pytorch, tensorflow, sklearn, xgboost and other libraries.
See full list: https://mlflow.org/docs/latest/python_api/index.html
Databricks platform: Databricks has a big list of technology partners which make using multiple vendors easier.
See full list: https://databricks.com/company/partners/technology"