Scheduling

Learn how to set, pause and stop a schedule for pipelines.

Schedules don't work for all orchestrators. Here is a list of all supported orchestrators.

Orchestrator	Scheduling Support	Supported Schedule Types
AirflowOrchestrator	✅	Cron, Interval
AzureMLOrchestrator	✅	Cron, Interval
DatabricksOrchestrator	✅	Cron only
HyperAIOrchestrator	✅	Cron, One-time
KubeflowOrchestrator	✅	Cron, Interval
KubernetesOrchestrator	✅	Cron only
LocalOrchestrator	⛔️	N/A
LocalDockerOrchestrator	⛔️	N/A
SagemakerOrchestrator	✅	Cron, Interval, One-time
SkypilotAWSOrchestrator	⛔️	N/A
SkypilotAzureOrchestrator	⛔️	N/A
SkypilotGCPOrchestrator	⛔️	N/A
SkypilotLambdaOrchestrator	⛔️	N/A
TektonOrchestrator	⛔️	N/A
VertexOrchestrator	✅	Cron only

Check out our tutorial on scheduling for a practical guide on how to schedule a pipeline.

Set a schedule

from zenml.config.schedule import Schedule
from zenml import pipeline
from datetime import datetime

@pipeline()
def my_pipeline(...):
    ...

# Use cron expressions
schedule = Schedule(cron_expression="5 14 * * 3")
# or alternatively use human-readable notations
schedule = Schedule(start_time=datetime.now(), interval_second=1800)

my_pipeline = my_pipeline.with_options(schedule=schedule)
my_pipeline()

Check out our SDK docs to learn more about the different scheduling options.

Update/Pause/Stop a schedule

You can update or delete your schedules using the following CLI commands:

# Update the cron expression of the schedule
zenml pipeline schedule update <SCHEDULE_NAME_OR_ID> --cron-expression='* * * * *'

# Delete a schedule
zenml pipeline schedule delete <SCHEDULE_NAME_OR_ID>

The functionality of these commands changes depending on whether the orchestrator that is running the schedule supports schedule updates/deletions:

• If the orchestrator supports it, this will update/delete the actual schedule as well as the schedule information stored in ZenML

• If the orchestrator does not support it, this will only update/delete the schedule information stored in ZenML

If the orchestrator does not support schedule management, maintaining the lifecycle of the schedule is the responsibility of the user. In these cases, we recommend the following steps:

1. Find schedule on ZenML

2. Match schedule on orchestrator side and delete

3. Delete schedule on ZenML

4. Re-run pipeline with new schedule

A concrete example can be found on the GCP Vertex orchestrator docs, and this pattern can be adapted for other orchestrators as well.