Orchestrators
How to orchestrate ML pipelines
The orchestrator is an essential component in any MLOps stack as it is responsible for running your machine learning pipelines. To do so, the orchestrator provides an environment which is set up to execute the steps of your pipeline. It also makes sure that the steps of your pipeline only get executed once all their inputs (which are outputs of previous steps of your pipeline) are available.
When to use it
The orchestrator is a mandatory component in the ZenML stack. It is used to store all artifacts produced by pipeline runs, and you are required to configure it in all of your stacks.
Orchestrator Flavors
Out of the box, ZenML comes with a local
orchestrator already part of the default stack that runs pipelines locally. Additional orchestrators are provided by integrations:
If you would like to see the available flavors of orchestrators, you can use the command:
How to use it
Inspecting Runs in the Orchestrator UI
If your orchestrator comes with a separate user interface (for example: Kubeflow, Airflow, Vertex), you can get the URL to the orchestrator UI of a specific pipeline run using the following code snippet:
Specifying per-step resources
Last updated