> For the complete documentation index, see [llms.txt](https://docs.zenml.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.zenml.io/stacks/stack-components/orchestrators/modal.md). # Modal Orchestrator The Modal orchestrator runs ZenML pipelines on [Modal](https://modal.com) using Modal Sandboxes. A Modal Sandbox is a remote container process: ZenML builds a Docker image for your pipeline, pushes it to your stack's container registry, and asks Modal to start that image in Modal's cloud infrastructure. {% hint style="info" %} Everywhere on this page, "sandbox" refers to Modal's Sandbox compute primitive (`modal.Sandbox`) — a Modal product concept, not a ZenML stack component. {% endhint %} The important runtime story is: 1. You start a ZenML pipeline run from your machine or CI system. 2. ZenML submits one Modal **orchestration sandbox** for the pipeline run. 3. That orchestration sandbox starts and monitors the step sandboxes that actually run your step code. 4. Step artifacts are read from and written to your ZenML artifact store. 5. ZenML stores the Modal sandbox IDs in run metadata so the run can be monitored and stopped later. This makes Modal useful when you want the whole pipeline, not just selected steps, to run away from your local machine while still using Modal's fast startup and resource selection. {% hint style="warning" %} This component is meant for stacks that use a ZenML server reachable from Modal. If your run depends on a local ZenML store or localhost-only services, the Modal sandbox will not be able to call back to your machine. {% endhint %} ## When to use it You should use the Modal orchestrator if: * you want your entire ZenML pipeline to run on Modal instead of locally. * your steps need Modal CPU, memory, GPU, cloud, or region settings. * you want both static and dynamic ZenML pipelines to run on Modal. * you already have remote ZenML stack components for artifacts, images, and code execution. If you only want a few selected steps to run on Modal while the rest of the pipeline is orchestrated somewhere else, use the [Modal step operator](/stacks/stack-components/step-operators/modal.md) instead. ## Requirements To use the Modal orchestrator, you need: * The ZenML `modal` integration installed: ```shell zenml integration install modal ``` * A [Modal account](https://modal.com/signup). * Modal authentication configured either with `modal setup`, Modal environment variables, or the `token_id` and `token_secret` fields on the ZenML orchestrator component. * Docker installed and running where ZenML builds images. * A [remote artifact store](/stacks/stack-components/artifact-stores.md). Modal cannot read artifacts from a local directory on your laptop. * A [remote container registry](/stacks/stack-components/container-registries.md). Modal pulls the Docker image from this registry. * An [image builder](/stacks/stack-components/image-builders.md) in your stack so ZenML can build the image that Modal runs. The stack validator rejects local artifact stores, local container registries, and stack components that expose local filesystem paths. The failure case is concrete: the Modal sandbox starts in Modal's infrastructure, asks for `/Users/you/project/artifacts/...`, and that path only exists on your machine. A remote artifact store and remote registry avoid that problem. {% hint style="warning" %} Modal imports images from a registry as `linux/amd64` images. If you build locally on Apple Silicon, configure your ZenML Docker settings to build `linux/amd64` images before using Modal. See the Docker build warning in the [Modal step operator docs](/stacks/stack-components/step-operators/modal.md) for an example configuration. {% endhint %} ## How to use it Register the orchestrator and add it to a stack with a remote artifact store, remote container registry, and image builder: ```shell zenml orchestrator register --flavor=modal \ --token_id= \ --token_secret= zenml stack register \ -o \ -a \ -c \ -i \ ... \ --set ``` If you already authenticated Modal with `modal setup`, or you provide Modal credentials through Modal's normal environment variables, you can omit `token_id` and `token_secret`. If you configure one of those fields on the orchestrator component, you must configure both. Then run your pipeline as usual: ```shell python run_pipeline.py ``` ZenML builds and pushes the image, submits the orchestration sandbox to Modal, and records the Modal sandbox metadata on the ZenML pipeline run. ## Static and dynamic pipeline behavior For a **static pipeline**, ZenML submits one Modal orchestration sandbox. Inside that sandbox, ZenML follows the step dependency graph and starts one child Modal sandbox for each step once its upstream steps have completed. If a step can be loaded from cache, the controller records the cached step run instead of starting a new Modal sandbox for that step. Static Modal pipelines support all three ZenML execution modes: `FAIL_FAST`, `STOP_ON_FAILURE`, and `CONTINUE_ON_FAILURE`. For a **dynamic pipeline**, ZenML submits one Modal orchestration sandbox that runs ZenML's dynamic pipeline entrypoint. Dynamic Modal pipelines still follow ZenML's current dynamic pipeline limitation: `CONTINUE_ON_FAILURE` behaves like `STOP_ON_FAILURE`, so after a dynamic step fails, ZenML does not keep launching later independent dynamic steps. Dynamic steps can run in two ways: * Steps that need isolation run as child Modal sandboxes. This includes steps with step-level resource settings or step-level Docker settings that differ from the pipeline image. Steps that use a step operator are isolated too, but they run on that step operator's infrastructure instead of a child Modal sandbox. * Steps that do not need isolation can run inside the dynamic orchestration sandbox process. In both modes, ZenML stores the orchestration sandbox ID on the pipeline run and child sandbox IDs on step runs. These metadata entries are what ZenML uses later for status checks and stop requests. ## Configuring Modal authentication The Modal orchestrator has optional `token_id` and `token_secret` fields. When both are configured, ZenML creates an explicit Modal SDK client from those credentials and passes that client to Modal SDK calls. When they are not configured, ZenML does not pass an explicit client. Modal then uses its normal authentication behavior, for example credentials from `modal setup`, Modal environment variables, or `~/.modal.toml`. ZenML also passes the short-lived ZenML server token needed by the runtime into Modal as a Modal Secret instead of a plain sandbox environment variable. Regular runtime environment variables are passed as normal sandbox environment variables. ## Configuring resources Use `ModalOrchestratorSettings` together with ZenML `ResourceSettings` to select Modal resources: ```python from zenml import pipeline, step from zenml.config import ResourceSettings from zenml.integrations.modal.flavors import ModalOrchestratorSettings gpu_step_settings = ModalOrchestratorSettings( gpu="A100", # GPU type, e.g. "T4" or "A100" # region="us-east-1", # optional; availability depends on your Modal plan # cloud="aws", # optional; availability depends on your Modal plan ) step_resources = ResourceSettings( cpu_count=2, memory="32GB", gpu_count=1, ) @step( settings={ "orchestrator": gpu_step_settings, "resources": step_resources, } ) def train_model() -> None: ... @pipeline def training_pipeline() -> None: train_model() ``` Orchestrator-wide options such as `modal_environment`, `timeout`, and `synchronous` can be set in a `ModalOrchestratorSettings` object passed at the pipeline level when you need them. {% hint style="warning" %} Set GPU types at the step level, not the pipeline level. Pipeline-level orchestrator settings configure the orchestration sandbox and are inherited by every step, so a pipeline-level `gpu` attaches a GPU to all step sandboxes and to the controller — where it sits idle while you pay for it, unless your dynamic pipeline runs GPU step code inside the orchestration sandbox process. {% endhint %} The resource mapping is: | ZenML setting | Modal sandbox setting | | -------------------------------------------------------------- | ----------------------------------------- | | `ResourceSettings.cpu_count` | `cpu` | | `ResourceSettings.memory` | `memory`, converted to MB and rounded up | | `ResourceSettings.gpu_count` + `ModalOrchestratorSettings.gpu` | `gpu`, for example `"A100"` or `"A100:2"` | | `ModalOrchestratorSettings.timeout` | `timeout` in seconds | | `ModalOrchestratorSettings.cloud` | `cloud` | | `ModalOrchestratorSettings.region` | `region` | Important details: * If `gpu_count > 0`, you must also set `ModalOrchestratorSettings.gpu`. ZenML needs the count and the Modal GPU type to build the Modal `gpu` argument. * If `gpu` is set and `gpu_count` is omitted, Modal uses one GPU of that type. * If `gpu` is set and `gpu_count=0`, ZenML runs on CPU and logs a warning that the GPU type was ignored. * Pipeline-level resources apply to the orchestration sandbox. * Static step-level resources apply to the child sandbox for that step. * Dynamic step-level resources cause the step to run as an isolated child sandbox, and the resources apply to that child sandbox. The `modal_environment` setting selects the Modal environment used for `modal.App.lookup(..., environment_name=...)`. It is separate from ZenML runtime environment variables. ## Synchronous and asynchronous runs By default, `synchronous=True`. In that mode, the process that submitted the run waits until the Modal orchestration sandbox finishes. If the controller detects failed child step sandboxes, it exits with a non-zero code, and ZenML reports the run as failed to the submitting process. If you set `synchronous=False`, ZenML submits the orchestration sandbox and returns after the sandbox has started successfully: ```python from zenml.integrations.modal.flavors import ModalOrchestratorSettings settings = { "orchestrator": ModalOrchestratorSettings(synchronous=False), } ``` The pipeline still runs on Modal. You can monitor it from ZenML because the pipeline run metadata contains the Modal orchestration sandbox ID. ## Stopping runs Stop behavior depends on the pipeline type and whether the stop is graceful or forceful. For a graceful stop of a static Modal run, ZenML marks the pipeline run as `STOPPING`. The orchestration sandbox notices that status, stops scheduling new step sandboxes, lets already-running child sandboxes finish, and then publishes the final run status. For a force stop of a static Modal run, ZenML terminates the Modal sandboxes it knows about: 1. ZenML refreshes the pipeline run metadata and reads known child sandbox IDs from step metadata and pipeline-run fallback metadata. 2. ZenML terminates each known child sandbox that is still running. 3. ZenML terminates the orchestration sandbox. 4. ZenML refreshes the run metadata once more and terminates any child sandbox IDs that appeared during cleanup. 5. If a sandbox has already finished, ZenML leaves it alone. For dynamic Modal runs, graceful stop requests use the same termination flow as force stop requests. ZenML terminates known child sandboxes and the orchestration sandbox, refreshes run metadata, and then terminates any late child sandboxes that appeared while cleanup was running. ## Current v1 limitations The first version of the Modal orchestrator has a few intentional limits: * Scheduled pipelines are not supported yet. Static and dynamic submissions with schedules are rejected. * The orchestrator does not handle step retries internally. ZenML's normal step retry behavior still applies. * Client-side caching is disabled for the orchestrator component. The static controller can still reuse already-cached steps when the ZenML run logic marks a step as cached. * Stop and status behavior depends on Modal sandbox IDs stored in ZenML metadata. If those IDs are missing, ZenML can only report the last known run or step status and may not be able to terminate already-started sandboxes. For more details on the available settings, see the [Modal integration SDK docs](https://sdkdocs.zenml.io/latest/integration_code_docs/integrations-modal.html).
ZenML Scarf
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.zenml.io/stacks/stack-components/orchestrators/modal.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.