Hooks
Run custom code on pipeline and step lifecycle events and record hook invocations.
Hooks let you run custom code at lifecycle points of a run or a step, and record those invocations as queryable HookInvocation records. The same machinery powers the built-in lifecycle hooks ZenML fires for you and the public API you can call from your own code. Common uses include sending notifications on success or failure, logging run details, and triggering external workflows.
Lifecycle hooks
Lifecycle hooks fire automatically and each fire creates one HookInvocation row.
on_start
Each execution attempt, before the step body
Once before the run starts
on_end
Each execution attempt, regardless of outcome
Once when the run reaches a terminal state
on_success
Once when the step completes successfully
Once when the run completes successfully
on_failure
Once when the step fails terminally
Once when the run fails
on_pause
—
Once when the run pauses
on_resume
—
Once when a paused run resumes
Step-level hooks fire for both static and dynamic pipelines, uniformly.
Static pipelines propagate, dynamic pipelines fire
The same @pipeline(on_*=...) kwarg has two different runtime meanings depending on whether the pipeline is dynamic.
@pipeline(on_*=X)
Static pipeline
Dynamic pipeline
on_start
Propagates to each step's on_start default
Fires once at the pipeline level
on_success
Propagates to each step's on_success default
Fires once at the pipeline level
on_failure
Propagates to each step's on_failure default
Fires once at the pipeline level
on_end
Propagates to each step's on_end default
Fires once at the pipeline level
on_pause
Ignored
Fires once at the pipeline level
on_resume
Ignored
Fires once at the pipeline level
For a static pipeline, a pipeline-level hook is a default that every step inherits where it has not set its own. No pipeline-level hooks are run. For a dynamic pipeline, the hook fires once at the run level and produces RUN_* entries. It fires on every run, including each invocation of a deployed pipeline. Dynamic pipeline users who want per-step defaults wire each @step directly.
Registering hooks
Pass a callable or a source string to the decorator, .configure(...), or .with_options(...).
Hook signatures
on_start, on_success, on_pause and on_resume take no arguments. on_failure and on_end optionally take a single BaseException argument.
exception is set only when the attempt or run failed. Details about the current step or run are available through the step or run context (see below).
Accessing step/run information in hooks
Step-scope hooks read the current step run and pipeline run through the step context. Run-scope hooks on a dynamic pipeline fire outside any step and read the run from the run context instead, via DynamicPipelineRunContext.get().run.
Sending alerts from hooks
Use the Alerter stack component to send notifications when a step or run fails or succeeds.
ZenML ships built-in alerter hooks for the common case.
Classifying failures and runs with TypeSafe Jev
The TypeSafe integration ships hooks that ask Jev, a fast and cheap classification model, typed questions about a failure or a finished run. Jev answers with a label or a probability instead of text, and the hooks store every answer as run metadata under keys starting with jev..
The hooks read the key from the TYPESAFE_API_KEY environment variable first and fall back to the api_key key of a secret named typesafe. With neither set, they log a warning and do nothing. API errors are also logged and never fail the step or run. The key and the other knobs (model, metadata key prefix, traceback length, baseline size) live on a JevConfig object, which keeps the key as a Pydantic SecretStr.
jev_failure_triage_hook classifies the exception and traceback of a failed step. It logs jev.failure_category (infrastructure, data, code, dependency or credentials, plus .confidence and .probabilities sub-keys) and jev.retry_likely_to_help (the probability that an unchanged rerun succeeds) on the failed step run:
Used as a run-level on_failure hook of a dynamic pipeline, it logs the same keys on the pipeline run instead.
jev_run_summary_hook is a run end hook for dynamic pipelines. When the run finishes, it sends Jev the run's status, duration and per-step results alongside a summary of the five most recent runs of the pipeline that finished before this one started, then logs jev.needs_attention (the probability that a human should look at the run) and jev.anomaly (0 means the run looks like recent runs, 2 means very unusual) on the pipeline run. On a static pipeline it skips with a warning, because static pipelines copy pipeline-level hooks onto each step and there is no finished run to assess.
To ask your own questions, call jev_classify_and_log from a custom hook. It logs the answers on the current step run, or on the pipeline run when called from a run-level hook of a dynamic pipeline:
Lifecycle hooks cannot take arguments, so to change what the built-in hooks ask or compare against, write a small hook of your own from the same building blocks. build_failure_state and build_run_summary_state produce exactly what the built-in hooks send to Jev, and FAILURE_TRIAGE_QUESTIONS and RUN_SUMMARY_QUESTIONS are the questions they ask. This hook adds a question to the failure triage:
JevConfig.load() accepts overrides for every field. This run summary hook compares against the last ten successful runs instead of the last five runs of any status, which stops a string of identical failures from looking normal:
For remote orchestrators, add the integration to the image with DockerSettings(required_integrations=["typesafe"]), and pass the API key as an environment variable or keep it in the typesafe secret.
Behavior notes
Retries. A retried step fires one
on_start/on_endpair per attempt.on_successandon_failurefire exactly once, at the terminal outcome.Cache hits. A cached step fires no step-level hooks. Pipeline-level hooks on a dynamic run still fire even when every step was cached.
Hook failures are swallowed. When a lifecycle hook raises, the run or step is not aborted. The exception is captured into the
HookInvocationrecord withstatus=FAILEDand execution proceeds.Async hooks. Hook functions can be defined with
async def. When the hook fires, ZenML runs the coroutine to completion and blocks until it finishes.Return values are discarded. Set
ZENML_TRACK_LIFECYCLE_HOOK_OUTPUTS=truein the execution environment to instead materialize lifecycle hook return values as output artifacts of the invocation, following the same rules asstore_return=Trueonrun_hook(see below).
Init and cleanup hooks
on_init and on_cleanup are pipeline setup and teardown hooks. They initialize and tear down shared run state rather than reacting to a single run or step outcome, so they follow different rules from the lifecycle hooks above. They are not recorded as HookInvocation records. When on_init fails, the run still records RUN_START, RUN_END, and RUN_FAILURE, but never a row for on_init itself. Find the root cause on pipeline_run.exception_info.
ZenML runs on_init once per execution environment, before any step body runs in that environment, and on_cleanup once when that environment is torn down. Where that lands depends on how the pipeline runs.
Deployments
on_init runs once per deployment replica when the replica starts, and on_cleanup once when it shuts down. Individual invocations of the deployed pipeline reuse the initialized state and do not re-run either hook. Lifecycle hooks like on_start and on_end still fire on every invocation.
Regular runs
For a run that is not a deployment, on_init runs once per execution environment:
Dynamic pipeline: once in the orchestrator environment, where the pipeline function executes.
Any step that runs outside the orchestration environment (static or dynamic): once ahead of that step body, the first time its environment is used. A step that shares the orchestration environment skips the hook, because the run context is already initialized there.
Recording custom invocations
Beyond the built-in lifecycle hooks, you can record arbitrary invocations from inside a step or a dynamic pipeline function. This is useful for instrumenting third-party callbacks such as the tool and model calls of an agent framework.
run_hook
Call run_hook(func, ...) to run a function and record the invocation in one call. The return value flows through to you.
Pass store_return=True to materialize the return value as an output artifact. A single unannotated return becomes one artifact named output. An annotated tuple return unpacks into one artifact per element.
Querying hook invocations
List the invocations recorded for a run through the client.
Last updated
Was this helpful?