Comment on page
Stacks: Configure MLOps Tooling and Infrastructure
How to configure MLOps tooling and infrastructure with stacks
This is an older version of the ZenML documentation. To read and view the latest version please visit this up-to-date URL.
In ZenML, a Stack represents a set of configurations for your MLOps tools and infrastructure. For instance, you might want to:
Any such combination of tools and infrastructure can be registered as a separate stack in ZenML. Since ZenML code is tooling-independent, you can switch between stacks with a single command and then automatically execute your ML workflows on the desired stack without having to modify your code.
In ZenML, each MLOps tool is associated to a specific Stack Component, which is responsible for one specific task of your ML workflow.
For instance, each ZenML stack includes an Orchestrator which is responsible for the execution of the steps within your pipeline, an Artifact Store which is responsible for storing the artifacts generated by your pipelines, and a Metadata Store that tracks what artifact was produced or consumed by which pipeline steps.
Check out the Categories of MLOps Tools page for a detailed overview of available stack components in ZenML.
As mentioned above, orchestrators, artifact stores, and metadata stores are the three components that need to be in every ZenML stack. The interaction of these three components enables a lot of the magic of ZenML, such as data and model versioning, automated artifact lineage tracking, automated caching, and more.

Orchestrators, Artifact Store, and Metadata Store
The Orchestrator is the component that defines how and where each pipeline step is executed when calling
pipeline.run()
. By default, all runs are executed locally, but by configuring a different orchestrator you can, e.g., automatically execute your ML workflows on Kubeflow instead.Under the hood, all the artifacts in our ML pipeline are automatically stored in an Artifact Store. By default, this is simply a place in your local file system, but we could also configure ZenML to store this data in a cloud bucket like Amazon S3 or any other place instead.
In addition to the artifact itself, ZenML automatically stores metadata about each pipeline run in a Metadata Store. By default, this uses an SQLite database on your local machine, but we could again switch it out for another storage type, such as a MySQL database deployed in the cloud.
The specific tool you are using is called a Flavor of the stack component. E.g., Kubeflow is a flavor of the Orchestrator stack component.
Out-of-the-box, ZenML already comes with a wide variety of flavors, which are either built-in or enabled through the installation of specific Integrations.
By default, ZenML itself and every Repository that you create already come with an initial active
default
stack, which features:If you followed the code examples in the Steps and Pipelines section, then you have already used this stack implicitly to run all of your pipelines.
You can see a list of all your registered stacks with the following command:
zenml stack list
Similarly, you can see all registered stack components of a specific type using:
zenml <STACK_COMPONENT> list
In order to see all the available flavors for a specific stack component, use:
zenml <STACK_COMPONENT> flavor list
Our CLI features a wide variety of commands that let you manage and use your stacks. If you would like to learn more, please run: "
zenml stack --help
" or visit our CLI docs.You can combine various MLOps tools into a ZenML stack as follows:
- 1.
- 2.
- 3.Activate the stack using
zenml stack set
. Now all your code is automatically executed using the desired tools / infrastructure.
First, you need to create a new instance of the respective stack component with the desired flavor using
zenml <STACK_COMPONENT> register <NAME> --flavor=<FLAVOR>
. Most flavors require further parameters that you can pass as additional arguments --param=value
, similar to how we passed the flavor.E.g., to register a local artifact store, we could use the following command:
zenml artifact-store register <ARTIFACT_STORE_NAME> \
--flavor=local \
--path=/path/to/your/store
In case you do not know all the available parameters, you can also use the interactive mode to register stack components. This will then walk you through each parameter (to skip just press ENTER):
zenml artifact-store register <ARTIFACT_STORE_NAME> \
--flavor=local -i
Afterwards, you should be able to see the new artifact store in the list of registered artifact stores, which you can access using the following command:
zenml artifact-store list
Our CLI features a wide variety of commands that let you manage and use your stack components and flavors. If you would like to learn more, please run
zenml <STACK_COMPONENT> --help
or visit our CLI docs.After registering each tool as the respective stack component, you can combine all of them into one stack using the
zenml stack register
command:zenml stack register <STACK_NAME> \
--orchestrator <ORCHESTRATOR_NAME> \
--artifact-store <ARTIFACT_STORE_NAME> \
--metadata-store <METADATA_STORE_NAME> \
...
You can use
zenml stack register --help
to see a list of all possible arguments to the zenml stack register
command, including a list of which option to use for which stack component.When you register a stack, the corresponding artifact- and metadata store gets coupled, which means that these instances can not be use in different pairings. This is an intended behavior to avoid mismatches between the two stores. In order get more information about this interaction, please refer to our CLI docs..
Finally, to start using the stack you just registered, set it as active:
zenml stack set <STACK_NAME>
Now all your code is automatically executed using this stack.
Some advanced stack component flavors might require connecting to remote infrastructure components prior to running code on the stack. This can be done using
zenml stack up
. See the Managing Stack States section for more details.If you have multiple stacks configured, you can switch between them using the
zenml stack set
command, similar to how you activate a stack.To unregister (delete) a stack and all of its components, run
zenml stack delete <STACK_NAME>
to delete the stack itself, followed by
zenml <STACK_COMPONENT> delete <STACK_COMPONENT_NAME>
to delete each of the individual stack components.
If you provisioned infrastructure related to the stack, make sure to deprovision it using
zenml stack down --force
before unregistering the stack. See the Managing Stack States section for more details.Last modified 1mo ago