Control execution order of steps
Last updated
Last updated
By default, ZenML uses the data flowing between steps of your pipeline to determine the order in which steps get executed.
The following example shows a pipeline in which step_3
depends on the outputs of step_1
and step_2
. This means that ZenML can execute both step_1
and step_2
in parallel but needs to wait until both are finished before step_3
can be started.
If you have additional constraints on the order in which steps get executed, you can specify non-data dependencies by passing the invocation IDs of steps that should run before your step like this: my_step(after="other_step")
. If you want to define multiple upstream steps, you can also pass a list for the after
argument when calling your step: my_step(after=["other_step", "other_step_2"])
.
Check out the documentation here to learn about the invocation ID and how to use a custom one for your steps.
This pipeline is similar to the one explained above, but this time ZenML will make sure to only start step_1
after step_2
has finished.