How to log and visualize ML experiments
Experiment trackers let you track your ML experiments by logging extended information about your models, datasets, metrics and other parameters and allowing you to browse them, visualize them and compare them between runs. In the ZenML world, every pipeline run is considered an experiment, and ZenML facilitates the storage of experiment results through Experiment Tracker stack components. This establishes a clear link between pipeline runs and experiments.
However, these ZenML mechanisms are meant to be used programmatically and can be more difficult to work with without a visual interface.
Experiment Trackers on the other hand are tools designed with usability in mind. They include extensive UI's providing users with an interactive and intuitive interface that allows them to browse and visualize the information logged during the ML pipeline runs.
You should add an Experiment Tracker to your ZenML stack and use it when you want to augment ZenML with the visual features provided by experiment tracking tools.
Here is an architecture diagram that shows how experiment trackers fit into the overall story of a remote stack.
Experiment Trackers are optional stack components provided by integrations:
If you would like to see the available flavors of Experiment Tracker, you can use the command:
zenml experiment-tracker flavor list
Every Experiment Tracker has different capabilities and uses a different way of logging information from your pipeline steps, but it generally works as follows:
- first, you have to configure and add an Experiment Tracker to your ZenML stack
- next, you have to explicitly enable the Experiment Tracker for individual steps in your pipeline by decorating them with the included decorator
- in your steps, you have to explicitly log information (e.g. models, metrics, data) to the Experiment Tracker same as you would if you were using the tool independently of ZenML
- finally, you can access the Experiment Tracker UI to browse and visualize the information logged during your pipeline runs
Note: the Expirement Tracker will declare run as failed if the pipeline step fails.