Skip to content
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Hooks

Run custom code on pipeline and step lifecycle events and record hook invocations.

Hooks let you run custom code at lifecycle points of a run or a step, and record those invocations as queryable HookInvocation records. The same machinery powers the built-in lifecycle hooks ZenML fires for you and the public API you can call from your own code. Common uses include sending notifications on success or failure, logging run details, and triggering external workflows.

Lifecycle hooks

Lifecycle hooks fire automatically and each fire creates one HookInvocation row.

Hook
Step scope
Pipeline scope (dynamic)

on_start

Each execution attempt, before the step body

Once before the run starts

on_end

Each execution attempt, regardless of outcome

Once when the run reaches a terminal state

on_success

Once when the step completes successfully

Once when the run completes successfully

on_failure

Once when the step fails terminally

Once when the run fails

on_pause

—

Once when the run pauses

on_resume

—

Once when a paused run resumes

Step-level hooks fire for both static and dynamic pipelines, uniformly.

Static pipelines propagate, dynamic pipelines fire

The same @pipeline(on_*=...) kwarg has two different runtime meanings depending on whether the pipeline is dynamic.

@pipeline(on_*=X)

Static pipeline

Dynamic pipeline

on_start

Propagates to each step's on_start default

Fires once at the pipeline level

on_success

Propagates to each step's on_success default

Fires once at the pipeline level

on_failure

Propagates to each step's on_failure default

Fires once at the pipeline level

on_end

Propagates to each step's on_end default

Fires once at the pipeline level

on_pause

Ignored

Fires once at the pipeline level

on_resume

Ignored

Fires once at the pipeline level

For a static pipeline, a pipeline-level hook is a default that every step inherits where it has not set its own. No pipeline-level hooks are run. For a dynamic pipeline, the hook fires once at the run level and produces RUN_* entries. It fires on every run, including each invocation of a deployed pipeline. Dynamic pipeline users who want per-step defaults wire each @step directly.

Registering hooks

Pass a callable or a source string to the decorator, .configure(...), or .with_options(...).

Hook signatures

on_start, on_success, on_pause and on_resume take no arguments. on_failure and on_end optionally take a single BaseException argument.

exception is set only when the attempt or run failed. Details about the current step or run are available through the step or run context (see below).

Accessing step/run information in hooks

Step-scope hooks read the current step run and pipeline run through the step context. Run-scope hooks on a dynamic pipeline fire outside any step and read the run from the run context instead, via DynamicPipelineRunContext.get().run.

Sending alerts from hooks

Use the Alerter stack component to send notifications when a step or run fails or succeeds.

ZenML ships built-in alerter hooks for the common case.

Classifying failures and runs with TypeSafe Jev

The TypeSafe integration ships hooks that ask Jev, a fast and cheap classification model, typed questions about a failure or a finished run. Jev answers with a label or a probability instead of text, and the hooks store every answer as run metadata under keys starting with jev..

The hooks read the key from the TYPESAFE_API_KEY environment variable first and fall back to the api_key key of a secret named typesafe. With neither set, they log a warning and do nothing. API errors are also logged and never fail the step or run. The key and the other knobs (model, metadata key prefix, traceback length, baseline size) live on a JevConfig object, which keeps the key as a Pydantic SecretStr.

jev_failure_triage_hook classifies the exception and traceback of a failed step. It logs jev.failure_category (infrastructure, data, code, dependency or credentials, plus .confidence and .probabilities sub-keys) and jev.retry_likely_to_help (the probability that an unchanged rerun succeeds) on the failed step run:

Used as a run-level on_failure hook of a dynamic pipeline, it logs the same keys on the pipeline run instead.

jev_run_summary_hook is a run end hook for dynamic pipelines. When the run finishes, it sends Jev the run's status, duration and per-step results alongside a summary of the five most recent runs of the pipeline that finished before this one started, then logs jev.needs_attention (the probability that a human should look at the run) and jev.anomaly (0 means the run looks like recent runs, 2 means very unusual) on the pipeline run. On a static pipeline it skips with a warning, because static pipelines copy pipeline-level hooks onto each step and there is no finished run to assess.

To ask your own questions, call jev_classify_and_log from a custom hook. It logs the answers on the current step run, or on the pipeline run when called from a run-level hook of a dynamic pipeline:

Lifecycle hooks cannot take arguments, so to change what the built-in hooks ask or compare against, write a small hook of your own from the same building blocks. build_failure_state and build_run_summary_state produce exactly what the built-in hooks send to Jev, and FAILURE_TRIAGE_QUESTIONS and RUN_SUMMARY_QUESTIONS are the questions they ask. This hook adds a question to the failure triage:

JevConfig.load() accepts overrides for every field. This run summary hook compares against the last ten successful runs instead of the last five runs of any status, which stops a string of identical failures from looking normal:

For remote orchestrators, add the integration to the image with DockerSettings(required_integrations=["typesafe"]), and pass the API key as an environment variable or keep it in the typesafe secret.

Behavior notes

  • Retries. A retried step fires one on_start / on_end pair per attempt. on_success and on_failure fire exactly once, at the terminal outcome.

  • Cache hits. A cached step fires no step-level hooks. Pipeline-level hooks on a dynamic run still fire even when every step was cached.

  • Hook failures are swallowed. When a lifecycle hook raises, the run or step is not aborted. The exception is captured into the HookInvocation record with status=FAILED and execution proceeds.

  • Async hooks. Hook functions can be defined with async def. When the hook fires, ZenML runs the coroutine to completion and blocks until it finishes.

  • Return values are discarded. Set ZENML_TRACK_LIFECYCLE_HOOK_OUTPUTS=true in the execution environment to instead materialize lifecycle hook return values as output artifacts of the invocation, following the same rules as store_return=True on run_hook (see below).

Init and cleanup hooks

on_init and on_cleanup are pipeline setup and teardown hooks. They initialize and tear down shared run state rather than reacting to a single run or step outcome, so they follow different rules from the lifecycle hooks above. They are not recorded as HookInvocation records. When on_init fails, the run still records RUN_START, RUN_END, and RUN_FAILURE, but never a row for on_init itself. Find the root cause on pipeline_run.exception_info.

ZenML runs on_init once per execution environment, before any step body runs in that environment, and on_cleanup once when that environment is torn down. Where that lands depends on how the pipeline runs.

Deployments

on_init runs once per deployment replica when the replica starts, and on_cleanup once when it shuts down. Individual invocations of the deployed pipeline reuse the initialized state and do not re-run either hook. Lifecycle hooks like on_start and on_end still fire on every invocation.

Regular runs

For a run that is not a deployment, on_init runs once per execution environment:

  • Dynamic pipeline: once in the orchestrator environment, where the pipeline function executes.

  • Any step that runs outside the orchestration environment (static or dynamic): once ahead of that step body, the first time its environment is used. A step that shares the orchestration environment skips the hook, because the run context is already initialized there.

Recording custom invocations

Beyond the built-in lifecycle hooks, you can record arbitrary invocations from inside a step or a dynamic pipeline function. This is useful for instrumenting third-party callbacks such as the tool and model calls of an agent framework.

run_hook

Call run_hook(func, ...) to run a function and record the invocation in one call. The return value flows through to you.

Pass store_return=True to materialize the return value as an output artifact. A single unannotated return becomes one artifact named output. An annotated tuple return unpacks into one artifact per element.

Querying hook invocations

List the invocations recorded for a run through the client.

Last updated

Was this helpful?