An end-to-end project

Put your new knowledge in action with an end to end project

This is an older version of the ZenML documentation. To read and view the latest version please visit this up-to-date URL.

An end-to-end project

In this guide, we will go over some advanced concepts:

• The value of deploying ZenML

• Abstracting infrastructure configuration into stacks

• Deploying an MLOps stack on a cloud provider of your choice

• Connecting a git repository

• Scaling compute

• Setting up pipeline configuration in production

We will now combine all of these concepts into an end-to-end MLOps project powered by ZenML.

Get started

Start with a fresh virtual environment with no dependencies. Then let's install our dependencies:

pip install "zenml[templates,server]" notebook
zenml integration install sklearn -y

We will then use ZenML templates to help us get the code we need for the project:

mkdir zenml_starter
cd zenml_starter
zenml init --template e2e_batch --template-with-defaults

# Just in case, we install the requirements again
pip install -r requirements.txt

Above doesn't work? Here is an alternative

The e2e template is also available as a ZenML example. You can clone it:

git clone [e-mail]:zenml-io/zenml.git
cd examples/e2e
pip install -r requirements.txt
zenml init

What you'll learn

The e2e project is a comprehensive project template to cover major use cases of ZenML: a collection of steps and pipelines and, to top it all off, a simple but useful CLI. It showcases the core ZenML concepts for supervised ML with batch predictions:

• Designing ZenML pipeline steps

• Using step parameterization and step caching to design flexible and reusable steps

• Using custom data types for your artifacts and writing materializers for them

• Constructing and running a ZenML pipeline

• Accessing ZenML pipeline run artifacts in the post-execution phase after a pipeline run has concluded

• Best practices for implementing and running reproducible and reliable ML pipelines with ZenML

Now try sharing the ZenML e2e template with your colleagues and see how they react!

Conclusion and next steps

The production guide has now hopefully landed you with an end-to-end MLOps project, powered by a ZenML server connected to your cloud infrastructure. You are now ready to dive deep into writing your own pipelines and stacks. If you are looking to learn more advanced concepts, the Advanced Guide is for you. Until then, we wish you the best of luck chasing your MLOps dreams!