Local Docker Orchestrator
How to orchestrate pipelines locally in Docker
You should use the local Docker orchestrator if:
- you want the steps of your pipeline to run locally in isolated environments.
- you want to debug issues that happen when running your pipeline in Docker containers without waiting and paying for remote infrastructure.
To use the local Docker orchestrator, we can register it and use it in our active stack:
zenml orchestrator register <ORCHESTRATOR_NAME> --flavor=local_docker
# Register and activate a stack with the new orchestrator
zenml stack register <STACK_NAME> -o <ORCHESTRATOR_NAME> ... --set
You can now run any ZenML pipeline using the local Docker orchestrator:
Note that if you wish to use this orchestrator to run steps on a GPU, you will need to follow the instructions on this page to ensure that it works. It requires adding some extra settings customization and is essential to enable CUDA for the GPU to give its full acceleration.