Local Docker Orchestrator

How to orchestrate pipelines locally in Docker

This is an older version of the ZenML documentation. To read and view the latest version please visit this up-to-date URL.

The local Docker orchestrator is an orchestrator flavor which comes built-in with ZenML and runs your pipelines locally using Docker.

When to use it

You should use the local Docker orchestrator if:

• you want the steps of your pipeline to run locally in isolated environments.

• you want to debug issues that happen when running your pipeline in Docker containers without waiting and paying for remote infrastructure.

How to deploy it

To use the local Docker orchestrator, you only need to have Docker installed and running.

How to use it

To use the local Docker orchestrator, we can register it and use it in our active stack:

zenml orchestrator register <ORCHESTRATOR_NAME> --flavor=local_docker

# Register and activate a stack with the new orchestrator
zenml stack register <STACK_NAME> -o <ORCHESTRATOR_NAME> ... --set

You can now run any ZenML pipeline using the local Docker orchestrator:

python file_that_runs_a_zenml_pipeline.py

Additional configuration

For additional configuration of the Local Docker orchestrator, you can pass LocalDockerOrchestratorSettings when defining or running your pipeline. Check out the API docs for a full list of available attributes and this docs page for more information on how to specify settings.

For more information and a full list of configurable attributes of the local Docker orchestrator, check out the API Docs.

Enabling CUDA for GPU-backed hardware

Note that if you wish to use this orchestrator to run steps on a GPU, you will need to follow the instructions on this page to ensure that it works. It requires adding some extra settings customization and is essential to enable CUDA for the GPU to give its full acceleration.