Use the Sandbox
Running pipelines in the MLOps Platform Sandbox.
This is an older version of the ZenML documentation. To read and view the latest version please visit this up-to-date URL.
Use the Sandbox
The MLOps Platform Sandbox is a temporary MLOps platform that includes ZenML, MLflow, Kubeflow, and Minio. Its purpose is to showcase the simplicity of creating a production-ready machine-learning solution using popular open-source tools. The Sandbox is designed for learning and experimentation, not for commercial projects or large-scale production use. It offers users a controlled environment to explore MLOps tools and processes before building their custom MLOps stack based on their specific requirements.
How does the Sandbox work?
After signing up, a user "creates a sandbox" which provisions the following services on a cluster managed by the ZenML team:
MLflow: An experiment tracker for tracking metadata.
Kubeflow: An orchestrator for running ML workloads on Kubernetes.
Minio: An artifact store for storing artifacts produced by pipelines.
ZenML: The MLOps framework that integrates everything.
Each sandbox comes with limited computing and storage resources. The ZenML service included with each sandbox has the following features:
A registered stack with other services, including all credentials as secrets.
A set of an example pipeline execution environment builds. These Docker images are hosted on Dockerhub and make running the example pipelines easy.
A connected code repository containing the code for the example pipelines. This repository is the official ZenML repository (with the code located in the examples directory).
In order to run the example pipelines, users need to download the repository locally and execute a pipeline with a supported build. The Sandbox UI offers a convenient interface for copying the relevant commands. The starter guide also provides more details on running these example pipelines.
How do I use the Sandbox to run custom pipelines?
As discussed above, the sandbox provides pre-built pipelines for users to run. If you want to try running these pipelines first, please visit this page in the starter guide to learn how to do this. The limitation on running custom pipelines is in place to control costs and demonstrate how MLOps engineers can enforce rules through a central control plane.
You might nevertheless be interested in using the resources provisioned in the Sandbox to run your own pipelines. There are two ways to do this:
Run pipelines without custom dependencies
If you have code that uses the same dependencies as the ones provided in the examples docs, you can simply update the example pipeline code (or even add new pipelines) by pushing to a git repository, and reusing existing environment builds of the example pipelines.
Of course, in order to be able to do that, you need to be able to push to the code repository. You can do this by following these steps:
Fork an existing Github repository or create a new one.
Push your code to the repository. Alternatively, you can copy the example code from the ZenML repository to your new repository and make the necessary modifications.
Register the new code repository with ZenML by following code repository guide.
Reuse the builds from the example code repository to execute your own code using the guide to build reuse.
Read more about how to connect a git repository to ZenML here.
After that is done, you can change the code and run the pipeline with your chosen execution environment build. Learn more about reusing execution environments here.
Please note that this will only work if you are using the same dependencies as the example code. If you are using different dependencies, such as a different framework or stack components, you will need to build and push your own Docker images to a container registry as described in the next section.
Run pipelines with custom dependencies
The sandbox is a pre-configured stack that connects to and uses ZenML's public container registry. This allows you to execute pipelines using the provided example code and dependencies, eliminating the need to build and push Docker images to a container registry on your own. However, if you wish to run your own code with custom dependencies, you must register a new stack that uses a public container registry where you have write access and the repository is publicly accessible so that Kubeflow can pull the images.
Let's take the Docker Hub registry as an example. To proceed, create a Docker Hub account and establish a new repository. Next, create a new stack component that uses this container registry. You can refer to the container registry stack component guide for detailed instructions.
Once the stack component is created, you can register a new stack that incorporates this component. Execute the following command:
Replace <MY_CONTAINER_REGISTRY> with the specific name of the container registry used in your stack
What to do when your sandbox runs out?
The Sandbox will only run for 4 hours. After that, it will be deleted. If you want to continue to use ZenML in a cloud deployment you can either:
Register a new Sandbox
Create and utilize a brand new Sandbox instance
Extend your Sandbox time limit
Fill out a form to extend the time limit of your Sandbox instances
Deploy your own cloud stack
Deploy and use a stack on a cloud environment
Last updated
