For the complete documentation index, see llms.txt. This page is also available as Markdown.

Server & SDK

Changelog for ZenML OSS and ZenML UI.

Stay up to date with the latest features, improvements, and fixes in ZenML OSS.

0.96.4 (2026-09-04)

See what's new and improved in version 0.96.4.

ZenML 0.96.4

Webhook-driven automation

  • Webhook triggers for pipeline snapshots: ZenML now supports first-class webhook-driven automation for secure, project-scoped integrations with GitHub or custom systems PR #5169. You can attach webhook triggers to pipeline snapshots and launch them when matching external events arrive, including typed GitHub filters for merged pull requests, completed workflow runs, pushes, and published releases.

  • ClickUp webhook provider: ZenML now includes a built-in ClickUp webhook provider for triggering automation from ClickUp task and list events PR #5216. The provider authenticates deliveries with ClickUp’s raw hex HMAC signature and supports string-based filters, so teams can connect ClickUp activity directly to ZenML workflows.

  • Slack webhook provider: ZenML Pro now supports inbound Slack Events API callbacks as a first-class webhook provider PR #5221. Slack apps can authenticate events such as mentions, messages, reactions, and other automation-focused activity to trigger pipeline snapshots or related workflows; this is separate from the existing Slack alerter that sends notifications to Slack.

  • AWS RDS IAM authentication for MySQL stores: The MySQL ZenML store now supports AWS RDS IAM database authentication while preserving the existing password-based behavior PR #5113. ZenML generates fresh IAM tokens for database connections and enforces TLS hostname and certificate verification in IAM mode, giving AWS users a passwordless database authentication option aligned with RDS security best practices.

  • Faster pipeline startup with reused build checksums: ZenML now precalculates and reuses Docker build checksums while resolving pipeline builds PR #5154. This avoids repeated build-configuration work during startup, especially for large pipelines with many steps, and also helps when skip_build=True because ZenML no longer repeats unnecessary build preparation checks.

  • Safer workload-scoped tokens: Workload-scoped API tokens can no longer be exchanged for persistent, unscoped API tokens PR #5152. This keeps credentials bound to the pipeline run, schedule, or deployment that created them, so workload permissions expire with the workload authorization instead of being escalated into longer-lived generic API access.

  • Production/stable package metadata: Future ZenML releases are now marked as Development Status :: 5 - Production/Stable on PyPI instead of beta PR #5186. This aligns the published package metadata with ZenML’s current maturity and makes the package classification clearer for users and automated tooling.

Fixed
  • Failed dynamic pipelines now report failure correctly: Dynamic pipelines running with ExecutionMode.STOP_ON_FAILURE no longer complete successfully when an isolated step fails PR #5182. ZenML now propagates the failed step state correctly so the overall pipeline run status matches the actual execution outcome.


0.96.3 (2026-08-07)

See what's new and improved in version 0.96.3.

ZenML 0.96.3

Runtime and orchestration

  • Multi-pod Kubernetes step operator jobs: Command steps can now run across multiple Kubernetes pods with the Kubernetes step operator. Set pod_count in KubernetesStepOperatorSettings to launch the step as an indexed job, making it easier to distribute command-style workloads across pods. PR #5104

  • Local Docker sandbox: ZenML now includes a local Docker sandbox, plus a unified settings model for containerized sandboxes. Local sandbox workflows also support file upload and download, making it easier to test containerized ZenML behavior locally before moving to remote infrastructure. PR #5102

Deployment, security, and artifact integrity

  • Enrollment keys from Kubernetes Secrets: The Helm chart now supports server.pro.enrollmentKeySecretRef, so ZenML Pro enrollment keys can be injected from an existing Kubernetes Secret instead of being stored inline in Helm release values. The secret reference is applied consistently to the server, migration, and worker containers. PR #5123

  • Cloudpickle artifact hash validation: Cloudpickle-materialized artifacts now store a SHA-256 hash when written and validate that hash before loading. This helps detect corrupted or unexpectedly modified artifact files earlier and fail with a clearer integrity signal. PR #5103

CLI and dashboard UX

  • Select a stack during login: zenml login now accepts --stack, allowing you to connect to a server and immediately set the active stack in one command. When used together with --project, ZenML applies the project first so the stack is resolved in the intended project context. PR #5125

  • Improved secret value display: Secret values in the dashboard no longer expand indefinitely in the UI and are truncated for readability. A direct copy action is now available, making it easier to work with long secret values without disrupting the page layout. PR #1104

  • Timeline updates for cancelled runs: The dashboard timeline now shows steps that were not started because a run was cancelled. The timeline filter also supports filtering for Not Started, making cancelled or partially executed runs easier to inspect. PR #1108

Fixed
  • Fewer SQLite lock failures for local stores: Local SQL stores backed by SQLite now wait up to 60 seconds for write locks instead of using Python’s 5-second default. This reduces sqlite3.OperationalError: database is locked failures when concurrent steps, such as mapped dynamic pipeline steps, finish and publish artifacts at the same time. PR #5096

  • Artifact deletion on deployed servers: Artifact version deletion now correctly applies project scope when checking whether an artifact is unused. This fixes deletion through the API on remote deployed servers where the default project is disabled. PR #5100

  • Complete exception tracebacks: Exception reporting now includes the full traceback lineage instead of only the final exception traceback. This makes chained failures easier to debug because the original cause is preserved alongside the final error. PR #5098

  • Smarter Kubernetes dynamic pipeline retries: Kubernetes orchestrator pods are no longer retried when the pipeline run is already in a finished state that cannot be retried. This avoids unnecessary pod restarts that would immediately exit with Run is already finished. PR #5107

  • Correct scoped prefix lookups: Prefix-based lookups now keep scoped and default filters properly constrained when matching by ID or name prefix. This prevents unrelated entities from being returned, for example when looking up a schedule trigger by a prefix that does not match its name or ID. PR #5126

  • Generic token provenance and service account auth: Generic and stack-deployment tokens now preserve the original service account or device provenance used to create them. This fixes authentication failures for pipeline workloads using server-issued JWTs for workspace-local service accounts, especially with external authentication enabled. PR #5127

  • MLflow tracking with managed runtimes and Databricks: ZenML’s MLflow experiment tracker now behaves more reliably when managed runtimes inject MLflow environment variables such as MLFLOW_RUN_ID. This avoids accidentally resuming an inherited run when ZenML needs to create its own run, improving compatibility with Databricks-backed MLflow setups. PR #5122

  • API keys after service account adoption: Existing workspace-level API keys remain valid when a workspace service account is adopted by an organization-level service account with the same name. This allows teams to migrate service accounts gradually without interrupting workloads that still use older API keys. PR #5138


0.96.2 (2026-07-17)

See what's new and improved in version 0.96.2.

ZenML 0.96.2

Dynamic pipelines

  • Explicit start ordering for dynamic steps: Dynamic pipelines now support start_after=... when calling steps, letting you control which concurrently launched steps should wait for others before starting. This makes it easier to model ordering constraints without turning concurrent parts of a dynamic pipeline into fully synchronous execution. Note that start_after is now a reserved step keyword, so steps that previously used a parameter with this name will need to be updated. PR #4995

  • More flexible dynamic step inputs: You can now configure whether JSON-serializable raw values passed to steps in dynamic pipelines should be treated as parameters instead of artifacts. The new environment-variable threshold defaults to 0 to preserve existing behavior, while explicit APIs such as with_options(parameters=...) and ExternalArtifact(...) remain available when you want to force either behavior. PR #5079

  • Improved dynamic execution semantics: Dynamic pipelines now support CONTINUE_ON_FAILURE execution mode, allowing already queued or asynchronous work to continue when an async step fails. ZenML also models implicit dependencies from newly launched steps to the last completed sync step, making mixed sync/async dynamic pipelines execute in a more predictable order. PR #5052

Integrations and deployment

  • DigitalOcean integration: ZenML now includes a first-class digitalocean integration with support for DigitalOcean Spaces artifact stores and DigitalOcean Container Registry stack components. Spaces support builds on the existing S3-compatible implementation while handling DigitalOcean regions and endpoint generation for you. PR #5054

  • Helm chart logging and OpenTelemetry configuration: The ZenML Helm chart now exposes server logging options and OpenTelemetry settings directly in values. You can configure console or JSON logging, service names, OTEL endpoints, and enable or disable traces, metrics, and logs without custom chart modifications. PR #5048

Data and metadata management

  • Server-side artifact data deletion: ZenML can now delete artifact version data through the server API, not only from a full client with direct stack access. This enables artifact metadata and backing data to be deleted from the UI or from thin clients that do not have the artifact store stack component locally available. PR #5034

  • Project metadata: Projects now support arbitrary project_metadata on create, update, and hydrated response models. Metadata is stored as portable JSON, preserved when omitted, replaced when explicitly supplied, and can be cleared by sending an empty object. PR #5086

Performance and scalability

  • Lower-memory cross-filesystem copies: fileio.copy() now streams cross-filesystem copies in bounded chunks instead of reading the entire file into memory. This significantly reduces peak memory usage for local-to-remote and remote-to-local artifact operations, including PathMaterializer, directory copies, integration materializers, and code archive upload/download flows. PR #5031

  • Fewer server requests during runs: ZenML now caches commonly reused project, store, stack, pipeline run, and completed step run responses in process where safe. Local and in-process execution paths make substantially fewer server requests, which improves responsiveness for pipelines with many steps. PR #5036 PR #5038

  • Faster DAG endpoint on large runs: The DAG endpoint now does less unnecessary parsing and object construction when serving large pipeline graphs. In benchmarks on a DAG with thousands of nodes and edges, endpoint latency was reduced by roughly half. PR #5051

Security and dependencies

  • FastAPI, Starlette, and OpenTelemetry updates: ZenML now supports FastAPI 0.138.0, raises the lower Starlette bound to 0.46.0 to pick up security fixes, and updates OpenTelemetry packages for compatibility. The update also removes an unused fastapi_utils dependency and cleans up deprecated FastAPI response and lifespan usage. PR #5017 PR #5060

Fixed
  • Token invalidation after credential changes: User tokens issued before a password change are now rejected, and tokens derived from API keys are tied to the key generation so rotated keys no longer leave stale sessions usable. The dashboard also correctly rotates API keys when a non-zero retention period is configured. PR #5025 PR #1088

  • Docker credentials supplied at runtime: ZenML now properly handles in-memory Docker credentials when a local Docker credential store already has entries for the same registry. This prevents the Docker Python client from silently preferring stale local credentials over credentials passed through ZenML. PR #5023

  • Remote image builds with restrictive .dockerignore files: ZenML no longer includes the root .dockerignore in generated build context archives for remote builders. This fixes builds on AWS CodeBuild, GCP Cloud Build, Kaniko, and similar builders when allowlist-style ignore patterns would otherwise exclude ZenML-generated files. PR #5033

  • Cleaner replay configurations: When replaying a pipeline run, ZenML now removes step parameters from the configuration if they are overridden by an input artifact. The displayed configuration no longer contains outdated values that were not actually used as step inputs. PR #5040

  • Deployment invocation with dict parameters: Dict-valued pipeline parameters sent to a deployment /invoke endpoint now replace compiled defaults instead of being recursively merged with them. This matches normal pipeline invocation behavior and prevents default keys from leaking into step inputs. PR #5042

  • Dynamic pipeline DAG race condition: ZenML now avoids a race where a step run could be visible in the database before its configuration was committed. This prevents intermittent 500 errors from the DAG endpoint while dynamic pipeline steps are being created. PR #5053

  • SSH orchestrator re-runs: SSH orchestrator container and Compose service names now avoid collisions across multiple runs of the same snapshot. Re-running from the dashboard, templates, or concurrent triggers no longer fails because an old container with the same name still exists on the host. PR #5082


0.96.1 (2026-07-02)

See what's new and improved in version 0.96.1.

ZenML 0.96.1
  • Run pipelines and steps over SSH: ZenML now includes an SSH orchestrator and SSH step operator for executing workloads on remote machines accessible via SSH (PR #4953). This makes it easier to use existing servers or on-prem infrastructure as execution targets without adopting a full cluster-based backend, while still managing runs through ZenML.


0.96.0 (2026-07-02)

See what's new and improved in version 0.96.0.

ZenML 0.96.0

Breaking Changes

  • The minimum supported transformers version has been raised. If you use ZenML with Hugging Face/transformers, update your environment and dependency pins to a newer compatible transformers release before upgrading ZenML. PR #4976

  • The Azure integration now requires newer Azure dependency versions, and support for the deprecated azureml-core library has been fully removed. If you use ZenML on Azure, update your Azure-related dependency pins and migrate any remaining azureml-core usage to the currently supported Azure SDK packages before upgrading. PR #4987

  • In open-source ZenML server deployments without RBAC enabled, service account and API key management is now restricted to admins only. Non-admin users will no longer be able to manage service accounts or API keys they previously created, so move any required credentials and automation to admin-managed accounts as part of your upgrade. PR #5007

New integrations and execution backends

  • Trackio experiment tracking: ZenML now includes a Trackio experiment tracker integration, allowing pipelines to log experiment data through Trackio’s public API. This makes it easier to manage trial results and connect ZenML runs with Hugging Face-backed Trackio workflows such as datasets, spaces, and buckets. PR #4841

  • Backblaze B2 artifact store: You can now configure Backblaze B2 as a ZenML artifact store. This adds another S3-compatible storage option for teams that want to store pipeline artifacts in Backblaze infrastructure. PR #4791

  • Baseten step operator: ZenML now supports a baseten step operator flavor for running GPU workloads as Baseten Training jobs. It supports regular single-node steps with ZenML artifacts and logs, as well as multi-node distributed training through command steps that can consume Baseten’s distributed training environment variables. PR #4973

  • Generic OAuth2 service connector: A new OAuth2 service connector lets you authenticate external services using a static token, client credentials, or a client ID with refresh token. This provides a reusable connector option for services that expose OAuth2-based authentication. PR #4992

Workflow controls and platform operations

  • Replay input overrides by step name: When replaying a run, you can now use step_default_input_overrides to override a step input for every invocation of a step with the same name. Per-invocation step_input_overrides still take precedence, giving you both broad and targeted control during replay. PR #4978

  • Trigger cycle protection: ZenML now detects execution loops in Platform Event Trigger chains at the pipeline level. Cyclic trigger dispatches are skipped with the new SKIPPED_TRIGGER_CYCLE status, while unrelated downstream dispatches can continue normally and the affected cycle can be inspected through the SDK. PR #4971

  • Optional sandbox cleanup on exit: Sandbox sessions can now be configured to destroy the sandbox automatically when the session exits. The option defaults to False, preserving the behavior from previous releases unless you opt in. PR #4986

  • Configurable Kubernetes API retries: Kubernetes-based deployments can now configure retry behavior for Kubernetes API calls. This gives operators more control over resilience in clusters where transient API failures or throttling can occur. PR #5004

  • Dashboard filtering and connector selection improvements: The dashboard Timeline View now has additional filtering options, including more status filters. Component creation also gets a more efficient connector selector, making setup flows smoother in larger workspaces. PR #1084

Performance and scalability

  • Faster pipeline sorting by latest run: Listing pipelines sorted by latest run is now more efficient on large deployments. ZenML changed the query shape and supporting database indexing so the server no longer has to scan all runs for all pipelines just to compute the latest run timestamp. PR #4969

  • More efficient run and artifact queries: Several common server queries now load only the data they need and fetch related metadata more efficiently. This improves performance for DAG, pipeline run, step run, artifact version, and model version views, especially in workspaces with many entities. PR #4994

  • Catch-up cleanup for expired API transactions: Expired API transaction cleanup now works through bounded catch-up passes instead of a single fixed delete per interval. This helps servers recover from cleanup backlogs while keeping each database operation bounded, and also allows completed expired idempotency transactions to be safely reclaimed. PR #4943

Fixed
  • Artifact store caching prevents server OOMs: The ZenML server now caches artifact store instances used for operations such as reading logs and visualizations. This avoids repeatedly rebuilding heavy storage clients and helps prevent memory growth that could previously lead to OOM kills on busy servers. PR #4974

  • Docker credentials for image builds and pushes: ZenML now configures Docker credentials correctly when building and pushing container images. This fixes cases where username/password credentials were not applied to the expected registry URI, which could cause authentication failures in build and push workflows. PR #5005

  • Secret backup and restore authorization: Secret backup and restore endpoints now enforce an explicit admin check when RBAC is disabled. This closes an authorization gap where an authenticated non-admin user could access admin-only secret operations in the default non-RBAC setup. PR #5009

  • Safer custom flavor loading: Custom flavor sources are now validated before server-side hydration. ZenML ensures the configured source resolves to a Flavor subclass before instantiation, preventing arbitrary zero-argument callables from being invoked during flavor loading. PR #5008


0.95.1 (2026-06-18)

See what's new and improved in version 0.95.1.

ZenML 0.95.1

Dynamic pipelines with step operators

Dynamic pipeline execution is more reliable when steps use step operators.

Fixed
  • Fixed an issue where running a step with a step operator in a dynamic pipeline could fail unless that step was explicitly listed in pipeline.depends_on.

  • ZenML now falls back to the orchestrator image in this case, matching the behavior already used for isolated steps without step operators. PR #4960

Faster pipeline and step run queries

Common pipeline run and step run views should now load more efficiently, especially on larger deployments.

  • Improved database query performance by adjusting how related data is loaded for common pipeline run and step run queries.

  • This helps avoid expensive query plans in MySQL for paginated run listings, making these queries more scalable. PR #4965

Logging stability

Logging shutdown is now safer when using artifact-backed log stores.

Fixed
  • Fixed a deadlock that could happen during logging context shutdown when using fsspec-based artifact log stores with debug logs enabled.

  • ZenML now avoids writing back into the log store during shutdown, improving reliability for pipeline and step log collection. PR #4964


0.95.0 (2026-06-17)

See what's new and improved in version 0.95.0.

ZenML 0.95.0

Breaking Changes

  • PR #4844: ZenML now supports Python 3.14, and environments using the local or server extras must also accommodate the SQLModel upgrade from 0.18.0 to 0.38.0. If you depend on those extras, review and update any pinned SQLModel-related dependencies before upgrading.

  • PR #4900: Local MLflow tracking now uses a SQLite backend by default when no tracking_uri is configured. New tracking metadata is stored in <LOCAL_ARTIFACT_STORE>/mlflow.db and artifacts under the local artifact store, so users relying on the previous default local MLflow layout or behavior should update their local setup and migration expectations.

  • PR #4790: ZenML now requires opentelemetry-sdk==1.40.0 instead of 1.38.0. If your environment pins OpenTelemetry packages, update them to compatible versions before upgrading ZenML.

  • PR #4875: Step and pipeline hooks have been reworked into a new lifecycle-based hook system with persisted hook invocation records. If you use hooks or related internal APIs, review your existing integrations and update them to the new hook semantics and lifecycle events.

  • PR #4919: ZenML server rate limiting no longer trusts raw X-Forwarded-For headers by default. If you run ZenML behind an ingress or reverse proxy, make sure proxy header handling is explicitly configured so login rate limiting continues to use the correct client IPs.

  • PR #4459: CLI list commands now return the newest items first by default instead of the oldest first. If you have scripts or workflows that assumed the previous ordering, update them to explicitly sort or handle the new default order.

  • PR #4566: The deprecated singular tag field has been removed from TaggableFilters. Update any API or client code to use the supported tag filtering format instead of passing a single tag value.

  • PR #4950: Pipeline execution may now raise different exception types depending on how step futures are awaited. If you catch exceptions around pipeline execution, review and update your error-handling logic to account for StepExecutionException being raised in implicit await scenarios.

  • PR #4867: ZenML now requires modal>=1.4.0,<2.0.0 when using the Modal integration.

New ways to run code and pipelines

This release expands how you can execute work in ZenML, from async Python to arbitrary commands and new remote execution backends.

  • Define steps and hooks with async def; ZenML now runs async functions on a fresh event loop for both normal and dynamic pipeline usage. PR #4913

  • Run arbitrary commands as pipeline steps with CommandStep(...), including non-Python commands and Python callables that do not require ZenML in the execution environment. PR #4904

  • Invoke deployments asynchronously: a new deployment endpoint can submit a pipeline run and return immediately instead of waiting for completion. PR #4906

Sandboxes and Modal execution

ZenML now includes the core sandbox abstraction for isolated execution, plus new backend support for Kubernetes and Modal-based workloads.

  • Added the core Sandbox stack component abstraction for running untrusted or generated code in isolated sessions, including a built-in local flavor for subprocess-based execution. PR #4866

  • Added a kubernetes sandbox flavor where each sandbox session runs in a dedicated Kubernetes pod, with streamed command execution and support for re-attaching to running sessions. PR #4926

  • Added a Modal orchestrator flavor so complete ZenML pipelines can run on Modal, using Modal sandboxes for orchestration and step execution. PR #4915

Integrations and deployment improvements

Several integrations and deployment paths are more flexible and production-ready.

  • Kubernetes deployments now merge pod_settings.resources into the deployment template context, making it possible to set pod resource limits required by cluster policies such as OPA Gatekeeper constraints. PR #4523

  • Databricks-managed MLflow deployments now support machine-to-machine OAuth authentication via service principals. PR #4947

Performance and scalability

Common list and hydration operations should be faster and more reliable on larger ZenML deployments.

  • Improved list endpoint ordering so descending sorts can use matching index scans instead of forcing expensive mixed-direction database sorts. PR #4890

  • Added targeted database indexes for common pagination and hydration query patterns across pipeline runs, snapshots, step configurations, step runs, and artifact versions. PR #4942

  • Adjusted request timeout behavior so only deduplicated/cacheable requests may return a timeout or backpressure response while work continues in the background. PR #4942

Security and permissions

This release tightens authorization checks around API keys, stack deployments, secrets, and tag-resource relationships.

  • Service-account API key validation now handles omitted internal verification values and client-provided key values consistently, while preserving internal re-authentication behavior. PR #4920

  • GET /api/v1/stack-deployment/stack now verifies READ permissions for both the returned stack and its associated service connector before returning deployment metadata. PR #4917

  • Secret reference resolution now prevents users from attaching private secrets owned by others, or internal ZenML-managed secrets, to their own resources. PR #4923

  • Tag-resource endpoints now require UPDATE permissions on the referenced resource before tag relationships can be created or deleted, including batch operations. PR #4927

  • Tag-resource RBAC enforcement now lives in the RBAC store layer for more consistent behavior, and tag reads remain broadly available as server-wide resources. PR #4938

Fixed
  • Fixed several dynamic pipeline edge cases around retries, stopping runs, and isolated step launch states:

    • Step failures that happen while launching a retry are now detected.

    • Steps no longer move to RETRYING if the run is already STOPPING or STOPPED.

    • Runs that fail while STOPPING now transition to STOPPED instead of FAILED.

    • Isolated steps now use PROVISIONING while they are being launched. PR #4916

  • Fixed an IndexError when step inputs annotated as bare list or tuple received multiple input artifacts. ZenML now loads each artifact using its stored data type, matching behavior for Any or unannotated inputs. PR #4929

  • Fixed GKE Kubernetes API endpoint selection in the GCP service connector by only using the DNS endpoint when it allows external traffic; otherwise, ZenML falls back to the IP-based endpoint. PR #4934


0.94.6 (2026-06-02)

See what's new and improved in version 0.94.6.

ZenML 0.94.6

Infrastructure & Deployment Improvements

  • Enhanced GKE Private Cluster Support: Fixed GCP service connector failures when connecting to private GKE clusters that use Google's DNS-based control plane endpoint. ZenML now connects using the same method as gcloud container clusters get-credentials --dns-endpoint, ensuring reliable access to private clusters. PR #4856

Fixed
  • Docker Build Requirements: Resolved an issue where ZenML would fail when no container engine was available, even when users explicitly set skip_build=True in their DockerSettings. The build checksum computation now correctly respects the skip_build flag. PR #4879


0.94.5 (2026-05-29)

See what's new and improved in version 0.94.5.

ZenML 0.94.5

🚀 Live Event Streaming for Pipeline Runs

You can now stream custom events in real-time from your running pipelines! Call zenml.streaming.publish() from inside any step or dynamic pipeline to push events that can be consumed via Server-Sent Events (SSE). Enable this feature by setting stream_broker_implementation_source in your server configuration. The initial implementation includes a Redis-based broker with automatic catch-up, gap signaling, and idle cleanup. PR #4804

📊 Pipeline Run Statistics Endpoint

A new POST /api/v1/runs/statistics endpoint lets you query aggregated metrics across your pipeline runs. Group by status, pipeline, stack, user, time buckets (hour/day/week/month), metadata values, tags, and more. Calculate averages, sums, min/max over duration, step counts, cached steps, output artifacts, or custom numeric metadata. Perfect for building dashboards and analytics. PR #4860

🎯 Richer Weights & Biases Integration

The W&B experiment tracker now automatically adds ZenML pipeline and step metadata to your W&B runs, groups runs by pipeline execution, and records W&B identifiers back to ZenML step metadata. You can now configure custom groups, job types, run configs, explicit or deterministic run IDs, resume behavior, and pass through additional wandb.init kwargs for complete control over your experiment tracking. PR #4838

🤖 Agentic Human-in-the-Loop Pipeline Example

A new example demonstrates building dynamic agentic pipelines with human approval gates. The example shows how to plan agent tasks, fan them out with step.map(), summarize results, pause execution with zenml.wait() for human review, and branch the final action based on the decision. Includes clean lineage tracking with Annotated step outputs. PR #4849

📝 Structured Logging and OpenTelemetry Instrumentation

The ZenML server now supports structured logging with OpenTelemetry instrumentation. Configure console output with the new ZENML_CONSOLE_LOGGING_FORMAT environment variable, choosing between console, json, or text formats. Server logs use a clean structured layout with timestamps, levels, logger context, and optional JSON fields. PR #4781

🛠️ Enhanced Developer Experience

  • String Type Annotations: Step and pipeline definitions now support string annotations, either explicitly quoted or via from __future__ import annotations. PR #4843

  • Build Cache Mounting: Python package installations during Docker image building can now mount a build cache for faster builds. PR #4820

  • Improved Wait Condition Input: When resolving wait conditions interactively with string schemas, you can now input raw strings without quotes—ZenML detects and handles this automatically. PR #4845

  • Better Logging Control: Restored custom console log formatting for non-DEBUG output, added step-name prefixes in terminal output during execution (disable with ZENML_DISABLE_STEP_NAMES_IN_LOGS=true), and kept stored logs clean and unformatted. The ZENML_LOGGING_FORMAT variable is now deprecated in favor of ZENML_CONSOLE_LOGGING_FORMAT. PR #4851

🎨 Dashboard Improvements

  • Improved scrolling behavior for the pipeline timeline view. PR #1053

Fixed
  • Pipeline runs now properly fail when the user pipeline function cannot be imported (e.g., due to missing packages), instead of staying in a running state indefinitely. PR #4832

  • Keyboard interrupts during active wait conditions are now handled more gracefully, preventing unclear error messages when the server transitions to STOPPED status. PR #4835

  • Fixed a copy-paste bug in the user update endpoint where admins couldn't activate or deactivate user accounts—the code was incorrectly writing is_admin instead of active to the safe update object. PR #4839


0.94.4 (2026-05-12)

See what's new and improved in version 0.94.4.

ZenML 0.94.4

New Databricks Step Operator

You can now run individual pipeline steps on Databricks using the new Databricks step operator PR #4648. This is useful when you want specific steps to execute in the Databricks runtime while the rest of your pipeline uses a different orchestrator. The Databricks orchestrator also now supports optional tag settings to label jobs and cluster resources for cost tracking, ownership, and governance.

Nested Dynamic Pipelines

Dynamic pipelines can now be nested, allowing you to call one dynamic pipeline from within another PR #4775. This enables more modular and reusable pipeline designs.

Enhanced Run:AI Training Workload Configuration

The Run:AI step operator now supports advanced training workload settings PR #4780, including:

  • Multiple mount types (PVC, ConfigMap, Secret, NFS, S3, HostPath)

  • Workload templates via workload_template_id

  • Security context settings (UID/GID, non-root execution, seccomp, capabilities)

  • Port declarations and external URL exposure

  • Training workload parallelism and completions

Improved Kubernetes Job Failure Diagnostics

When dynamic pipeline jobs fail due to system issues (such as OOM kills), ZenML now provides richer diagnostic information PR #4800. This makes it easier to understand why Kubernetes terminated your pods.

Better Kubernetes Label Handling

Kubernetes string handling has been improved with separate sanitization for DNS-style names/keys and looser label-value rules for metadata like run, pipeline, and step IDs PR #4756. This makes it easier to navigate through runs in Kubernetes.

Increased Secret Size Limit

The maximum allowed size for ZenML secrets stored in the SQL secrets store has been increased to 64KB PR #4769. The limit applies to the combined size of all keys and values in a secret object.

Dashboard: Parent Run Display

The dashboard now displays parent run information in the run details view when available PR #1050.

Fixed
  • Signal handling during step execution: Signal handlers are now properly unregistered after step execution, preventing strange errors when running many sync steps or interrupting pipelines PR #4784. Dynamic pipeline steps running in isolated environments or step operators are no longer affected by signal handling from the orchestrator.

  • RBAC performance: Fixed redundant RBAC permission checks during response model dehydration PR #4797. Previously, when permissions were already prefetched and denied, additional RBAC requests were sent unnecessarily for each sub-model.

  • Keyword-only arguments in steps: Step functions can now use keyword-only arguments without causing failures PR #4798.

  • Kubernetes orchestrator settings: Fixed step pod configuration by replacing legacy hardcoded orchestrator.kubernetes lookups with proper orchestrator.get_settings(...) calls PR #4803. Step pods now correctly apply orchestrator settings from canonical component keys while maintaining backward compatibility.


0.94.3 (2026-04-24)

See what's new and improved in version 0.94.3.

ZenML 0.94.3

🚀 New Features

Podman Support & Container Engine Abstraction

ZenML now supports Podman as an alternative to Docker for container image management. A new ContainerEngine abstraction has been introduced to make it easier to work with different OCI-compatible container runtimes. This gives you more flexibility in choosing your container tooling, especially in environments where Docker isn't available or preferred. PR #4651

Resource Pools (Pro only)

Introducing Resource Pools - a new way to manage and organize compute resources in ZenML. This feature includes full SDK methods, CLI commands, and API endpoints to create and manage resource pools and their associated objects, giving you better control over resource allocation across your ML workflows. PR #4465

Platform Event Triggers (Pro only)

You can now set up event-based triggers that automatically execute downstream workflows based on ZenML platform events, such as when a pipeline run completes. This enables powerful automation patterns and reactive workflows without manual intervention. Full backend management including CLI and SDK support is now available. PR #4692

Server-Side Pipeline Replays (Pro only)

Pipeline runs can now be replayed from the server with advanced capabilities:

  • Skip specific steps - steps won't re-execute even if cache was disabled or inputs changed

  • Override step input artifacts - go beyond parameter overrides and replace inputs that came from upstream steps in the original run

This gives you fine-grained control over re-executing parts of your pipelines. PR #4716

Multiple Components Per Stack Type

Stacks can now include multiple components of the same type for Alerters, Step Operators, and Experiment Trackers. You can designate one as the default while having others available for specific use cases, providing more flexibility in stack composition. PR #4671

Kubernetes Gateway API Support

The ZenML server can now be exposed via Gateway API HTTPRoute resources as an alternative to Ingress. Both Ingress and Gateway can be enabled simultaneously, making it easy to migrate from Ingress to Gateway API with zero downtime. PR #4726

✨ Enhancements

Improved Trigger Management

  • Triggers now track per-snapshot dispatch state showing whether the last run succeeded, was skipped due to concurrency policy, or failed

  • Failed dispatches store richer error context including message, type, severity, stack trace, and timestamps

  • New acknowledge flow lets you clear stored error context without re-attaching triggers or changing configuration

PR #4743

Schedule Stop Criteria

Schedules now support a max-runs limit to automatically stop after a specified number of executions per snapshot, giving you better control over scheduled pipeline runs. PR #4752

Non-Blocking Concurrent Steps

Dynamic pipelines with concurrent steps now execute more efficiently. Waiting for step inputs happens asynchronously instead of blocking the main thread, significantly improving pipeline execution performance when using concurrent steps. PR #4699

🐛 Fixed
  • Run status updates: Fixed a bug where pipeline runs weren't properly marked as failed when a step pod failed after initially reporting success PR #4747

  • Secret visibility updates: You can now properly update secrets from private to public using the CLI or client PR #4755

  • Logs for restarted runs: Removed the unique constraint from logs schema to allow log entries for repeated executions of the same pipeline run PR #4729

  • Air-gapped performance: Flavor model generation no longer calls PyPI to resolve ZenML versions, improving performance in air-gapped environments PR #4744


0.94.2 (2026-04-08)

See what's new and improved in version 0.94.2.

ZenML 0.94.2

🎨 Dashboard Enhancements

The ZenML dashboard now includes a Run Summary View that provides a comprehensive overview of your pipeline runs at a glance PR #1029. Timeline rows now automatically resize for better visualization of your pipeline execution history PR #1028.

🔧 Pipeline & Trigger Improvements

Artifact Name Substitutions for Dynamic Pipelines: You can now use artifact name substitutions in dynamic pipelines, making it easier to reference and manage artifacts programmatically PR #4668.

Enhanced Trigger Configuration: Triggered runs now follow a cleaner snapshot.source pattern. You can provide a configuration object to customize parameters for all triggered runs in an attachment, giving you more control over automated pipeline executions PR #4610.

⚠️ Deprecation Notice

Helm Chart Configuration Update: The top-level zenml: values key in the ZenML Helm chart has been renamed to server: PR #4637. Your existing values files will continue to work — the chart automatically merges both keys for backwards compatibility. If both keys are present, zenml: takes precedence for overlapping fields. The zenml: key is deprecated and will be removed in a future release; we recommend migrating to server: in your Helm configurations.

Fixed
  • Schedule Management: Fixed issues where deletion by ID prefix wasn't working for archived schedules, and reusing an archived schedule name was incorrectly blocked. Documentation has been updated to clarify schedule behavior post-deletion, including archived schedule renaming, hard deletion guidelines, and discovery of archived objects PR #4641.

  • Input Artifact Handling: Resolved a bug where a list of input artifacts containing a single item was incorrectly treated as a scalar input artifact, causing materializer type incompatibility errors PR #4679.

  • HTTP Request Optimization: The system now dynamically reduces chunk size when fetching many steps to avoid HTTP 414 (URI Too Long) errors PR #4553.

  • Dashboard Deployment Stability: Fixed a template loading issue in the FastAPI deployment app's dashboard that could cause TypeError: unhashable type: 'dict' on certain environments (e.g., Kubernetes with newer Jinja2 versions), preventing the server from starting when the dashboard was enabled PR #4649.


0.94.1 (2026-03-19)

See what's new and improved in version 0.94.1.

ZenML 0.94.1

🎯 Pipeline Execution Control

  • Pause and Resume Pipeline Runs: Introduced zenml.wait(...) to pause dynamic pipelines while waiting for external inputs, automatically freeing resources until the input is provided. Runs can be resumed automatically (when using remote orchestrators with snapshot support) or manually via zenml pipeline runs resume <ID>. PR #4588

  • Override Step Inputs on Replay: You can now override step inputs when replaying a pipeline run, giving you more flexibility to rerun pipelines with different data. PR #4590

🔧 Materializers and Data Handling

  • Dataclass Materializer: Added a built-in materializer for JSON-serializable dataclasses, making it easier to pass structured data between steps. PR #4600

  • LakeFS Data Versioning Example: New example demonstrating the "pass references, not data" pattern for terabyte-scale datasets. ZenML steps exchange lightweight LakeFS pointers while actual data stays in LakeFS, accessed via its S3-compatible gateway. PR #4559

☁️ Infrastructure and Deployment

  • Helm Environment Variable Overrides: Environment variables specified in zenml.environment, zenml.secretEnvironment, and worker deployment configurations can now override computed settings from the Helm chart. PR #4595

  • Secret Environment Variables in Helm: Added support for injecting secret environment variables into the ZenML server deployment via Helm without committing secrets to values.yaml, enabling better GitOps workflows. PR #4606

  • Docker Build Arguments: Build arguments defined in DockerSettings are now properly declared with ARG instructions in auto-generated Dockerfiles. PR #4612

🔐 Authentication and Credentials

  • Improved GCP Credentials Refresh: Implemented native GCP credentials refresh using service connector logic, replacing periodic expiration checks for more reliable OAuth2 credential handling. PR #4527

📊 Pipeline Configuration

  • Step Parameter Schema Storage: Step parameter specifications are now stored for better schema validation when triggering pipeline snapshots. PR #4591

Fixed
  • Exception Handling for Isolated Steps: Isolated steps in dynamic pipelines now raise the correct exception types instead of always wrapping them in RuntimeError, making error handling more intuitive and consistent. PR #4589

  • Kubernetes API Request Timeout: Fixed typing issue in the Kubernetes orchestrator and step operator to ensure api_request_timeout is properly applied as an integer value. PR #4605

  • Client-Server Compatibility: Resolved compatibility issue with clients <=0.92.0 by not passing both pipeline run ID and step run ID to logs requests. PR #4614

  • Dynamic Pipeline Monitoring: Fixed dictionary size change error that occurred when the main thread modified the _steps_to_monitor dictionary while the monitoring thread was iterating over it. PR #4619

  • Step Config Template Parameters: Step parameters defined in config templates now correctly take precedence over function default values in dynamic pipelines. PR #4624

📦 Dependencies

  • Updated Pydantic to version 2.12.5. PR #4552


0.94.0 (2026-03-04)

See what's new and improved in version 0.94.0.

ZenML 0.94.0

Breaking Changes

  • Old endpoints and client methods for legacy triggers, actions and event sources have been removed. This shouldn't affect you unless you explicitly used those endpoints or methods in your code.

  • Custom step operator flavors must implement new submit_step and get_step_status methods to work with dynamic pipelines. The legacy launch method will only work in static pipelines as a fallback. The Spark step operator is not yet compatible with dynamic pipelines. PR #4515

🚀 New Integrations

  • Run:AI Step Operator: ZenML now supports running individual pipeline steps on Run:AI clusters with fractional GPU allocation, enabling more efficient resource utilization for ML workloads. PR #4439

✨ New Features

  • Step and Pipeline Replays: You can now replay existing step or pipeline runs with the same inputs and configuration. When replaying a pipeline run, you can specify which steps to skip and reuse from the original run. A debug mode is also available to run replays on your active stack with a local orchestrator. PR #4456

  • Triggers and Native Schedules (PRO): Introduced the Trigger concept for automated pipeline execution. The first supported trigger type is Schedules, which offers lifecycle management, automatic synchronization with orchestrators, and centralized management across stacks. PR #4482

  • Step Run Filtering by Version: Added the ability to filter step runs by version, making it easier to track and manage specific versions of your pipeline steps. PR #4518

🔧 Improvements

  • Enhanced Dynamic Pipeline Monitoring: Improved the execution and monitoring of isolated steps in dynamic pipelines. Step submission is now separated from monitoring, preventing thread blocking during step execution. PR #4369

  • SkyPilot Integration Update: Updated the SkyPilot integration to support version 0.11.x, including migration to the new async API and support for new resource settings. PR #4462

  • Kubernetes Retry Configuration: Added configurable timeout options for Kubernetes orchestrator and step operator API calls, ensuring proper retry behavior and preventing unnecessary hanging. PR #4525

  • Git Submodule Support: Code archives now include files from git submodules when uploading to the artifact store, ensuring complete code tracking for repositories with submodules. PR #4496

Fixed
  • Fixed handling of variadic keyword arguments in pipeline functions, ensuring they are properly flattened before being passed to the pipeline. PR #4528

  • Fixed Azure integration dependencies by limiting the azure-mgmt-resource version to avoid compatibility issues with the 25.0.0 package split. PR #4516

  • Fixed handling of external artifacts with None values and reduced chunk size when fetching many step runs to avoid URI too large errors. PR #4551


0.93.3 (2026-02-19)

See what's new and improved in version 0.93.3.

ZenML 0.93.3

Performance Improvements

This release includes significant performance optimizations for the ZenML server, particularly when handling large-scale deployments:

  • Improved database query efficiency: Rewrote filtering queries to eliminate unnecessary sorting during item counting, removed inefficient DISTINCT statements on multiple columns, and optimized OR subqueries for better database performance at scale. PR #4449

  • Enhanced API transaction management: Moved cleanup of expired transactions to an independent background thread that runs periodically, significantly improving API response times especially for large payloads like pipeline snapshots with many steps. PR #4453

Logging Enhancements

Logging capabilities have been expanded with new features and improvements:

  • Added new create and update endpoints for logs with support for UUIDs in StepRunRequest and PipelineRunRequest

  • Introduced workspace ID and name to pipeline run log metadata (with backward compatibility)

  • Added zenml.event.type to error messages for better context tracking

  • Introduced environment variable to manage maximum log entries per request

  • Fixed inconsistent metadata key formatting (standardized zenml. prefix) PR #4405, PR #4467

Dashboard Updates

  • Added elapsed time display to step nodes in the DAG visualization for better pipeline monitoring PR #994

Fixed
  • Critical data loss bug: Fixed a critical issue in download_artifact_files_from_response that caused silent data corruption when downloading artifacts larger than 8KB. The bug resulted in up to 98%+ data loss for large artifacts by only preserving the last chunk of data. PR #4422

  • ZenML Pro migration: Fixed an issue where cookies from local user accounts persisted after migrating a ZenML OSS server to ZenML Pro via organization enrollment, preventing access to migrated resources in the UI. The server now properly rejects these stale cookies. PR #4473

  • UV-only environments: Fixed pipeline run crashes in environments using only uv without pip installed. ZenML now falls back to uv pip freeze when pip freeze is unavailable for collecting environment metadata. Also added UV_FREEZE as an export method for Docker builds. PR #4484

  • Kubernetes credential expiration: Fixed an issue where Kubernetes credentials issued by service connectors expired while monitoring long-running jobs, causing monitoring failures. Credentials are now properly refreshed during job monitoring. PR #4493

  • Improved CLI messaging when attempting to activate a stack without proper permissions


0.93.2 (2026-01-29)

See what's new and improved in version 0.93.2.

ZenML 0.93.2

🎨 Dashboard Enhancements

The ZenML Dashboard now provides better visibility into your pipelines and infrastructure:

  • Download Pipeline Code: You can now download the code used for a pipeline snapshot directly from the dashboard. A new Download button appears in the "Code Path" section on both the Pipeline Run details page and the Step details sheet, making it easy to retrieve and review the exact code that was executed. PR #4401, PR #989

  • Exception Information Display: When dynamic pipeline runs fail, the dashboard now displays detailed exception information, helping you quickly diagnose and troubleshoot issues. PR #4395, PR #990

  • Stack & Component Labels: Labels attached to stacks and components are now visible in the dashboard, making it easier to organize and identify your infrastructure resources. PR #992

🔄 Dynamic Pipeline Improvements

Dynamic pipelines are now more robust and easier to work with:

  • Proper Environment Configuration: The pipeline environment is now correctly set while running the entrypoint function of dynamic pipelines, ensuring consistent behavior across different execution contexts. PR #4420

🤖 Developer Experience

  • Claude Code Plugin: A new ZenML Quick Wins skill for Claude Code helps you implement MLOps best practices directly in your AI-assisted coding workflow. The plugin is available through the Claude Code plugin marketplace and includes comprehensive documentation for multiple AI coding tools. PR #4426

Fixed

🚀 Performance & Scalability

  • Artifact Download Fix: Resolved an issue where artifact version downloads were failing due to incorrect RBAC checks on the download endpoint. PR #4401


0.93.1 (2026-01-14)

See what's new and improved in version 0.93.1.

ZenML 0.93.1

🎛️ Schedule Management Enhancements

You can now pause and resume pipeline schedules directly from the CLI, giving you better control over automated pipeline executions. Use the new commands to activate or deactivate schedules on demand:

Currently available for the Kubernetes orchestrator. PR #4328

Schedules now support archiving as a soft-delete operation. When you delete a schedule, it's archived instead of permanently removed, preserving historical references so your pipeline runs maintain their schedule associations. PR #4339

🖥️ Dashboard Improvements

Stack Management: You can now update existing stacks directly from the UI without having to delete and recreate them. A new dedicated stack update page lets you add or replace stack components (orchestrators, artifact stores, container registries, etc.) efficiently. PR #978

Step Cache Management: View and manage step cache expiration directly from the step details panel. The cache expiration field shows when a step's cache will expire (or "Never" if no expiration is set), with expired caches clearly marked. You can also manually invalidate a step's cache with a single click. PR #976

Enhanced Logs Experience: Pipeline runs now have a dedicated logs page with a sidebar for navigating between run-level and step logs. The new logs viewer features virtualized rendering for better performance with large outputs, search and filtering capabilities, and step duration display. PR #985

⚡ Performance & Reliability

Kubernetes Orchestrator Improvements: The Kubernetes orchestrator now runs more efficiently with configurable DAG runner workers, optimized cache candidate fetching, and better error handling for failed step pods. PR #4368

Database Backup Speed: A new mydumper/myloader backup strategy delivers dramatically faster operations:

  • 30x faster database backups

  • 2.5x faster database restores

  • 10x lower storage space requirements

PR #4358

🚀 Orchestrator Features

AzureML Dynamic Pipelines: Dynamic pipelines are now fully supported on the AzureML orchestrator, expanding your options for flexible pipeline execution. PR #4363

Kubernetes Init Container Templating: When configuring init containers for the Kubernetes orchestrator, you can now use an "{{ image }}" placeholder that will be automatically replaced with the actual orchestration/step container image. PR #4361

Fixed
  • Fixed per-step compute settings not being applied correctly PR #4362

  • Fixed database migration script to handle pipelines with zero runs PR #4360

  • Fixed working directory in dynamic pipeline containers (was /zenml instead of /app) PR #4379

  • Fixed pipeline run status updates in CONTINUE_ON_FAILURE execution mode PR #4379

  • Fixed component setting shortcut keys when running snapshots PR #4379

  • Improved error messages during source validation and for string type annotations PR #4359

  • Fixed log storage in Kubernetes orchestrator by propagating context vars to DAG runner threads PR #4359

  • Pipeline source code now included for runs triggered by snapshots/deployments PR #4359


0.93.0 (2025-12-16)

See what's new and improved in version 0.93.0.

ZenML 0.93.0

Breaking Changes

  • The logging system has been completely redesigned with a new log store abstraction that now captures stdout, stderr, and all logger outputs more comprehensively. If you have custom integrations that relied on the previous logging behavior or accessed logs directly from the artifact store, you may need to update your code to use the new log store APIs. PR #4111

  • The REST API endpoint /api/v1/pipelines/<ID>/runs has been removed. Use /api/v1/runs?pipeline_id=<ID> instead to fetch runs for a specific pipeline. PR #4350

  • The logs field has been removed from the response models of pipeline runs and steps. Additionally, RBAC checks for fetching logs, downloading artifacts, and visualizations have been tightened. If you were accessing logs through these response models, you will need to use the dedicated log fetching endpoints instead. PR #4347

Enhanced CLI Experience

The ZenML CLI now provides a more flexible and user-friendly experience with improved table rendering and output options. Tables are now more aesthetically pleasing with intelligent column sizing, and you can pipe CLI output in multiple formats (JSON, YAML, CSV, TSV) by properly separating stdout and stderr streams. This makes it easier to integrate ZenML commands into scripts and automation workflows. PR #4241

Dynamic Pipeline Support

Dynamic pipelines can now be deployed and run with the local Docker orchestrator, including support for asynchronous execution. This expands the flexibility of local development and testing workflows, allowing you to leverage dynamic pipeline patterns without requiring cloud infrastructure. PR #4294, PR #4300

Pipeline Run Tracking

Each pipeline run now includes an index attribute that tracks its position within the pipeline's execution history, making it easier to identify and reference specific runs in a sequence. PR #4288

Orchestrator Health Monitoring

The Kubernetes orchestrator now includes enhanced health monitoring capabilities with configurable heartbeat thresholds. Steps that become unhealthy are preemptively stopped, and pipeline tokens are automatically invalidated when pipelines enter an unhealthy state, improving reliability and resource management. PR #4247

New Integrations

  • Alibaba Cloud Storage: Added support for Alibaba Cloud OSS as an artifact store, expanding ZenML's cloud storage options. PR #4289

  • Generic OTEL Log Store: Introduced a new log store flavor that can connect to any OTEL/HTTP/JSON compatible log intake endpoint, enabling integration with a wider range of observability platforms. PR #4309

Azure ML Enhancements

The AzureML orchestrator and step operator now support shared memory size configuration, giving you more control over resource allocation for your workloads. PR #4334

Fixed
  • MLflow Experiment Tracker: Fixed crashes when attempting to resume non-existent runs on Azure ML. The tracker now validates cached run IDs and gracefully creates new runs when necessary. PR #4227

  • Kubernetes Service Connector: Resolved failures in the ZenML server related to the Kubernetes service connector caused by incompatible urllib3 and kubernetes client library versions. PR #4312

  • Datadog Log Store: Improved log fetching with proper pagination support, handling the Datadog API's 1000-log limit per request through cursor-based iteration. PR #4314

  • Deployment Log Flushing: Eliminated blocking behavior when flushing logs during deployment invocations, preventing potential hangs at pipeline completion. PR #4354


0.92.0 (2025-12-02)

See what's new and improved in version 0.92.0.

ZenML 0.92.0

Dynamic Pipeline Support Expansion

This release significantly expands support for dynamic pipelines across multiple orchestrators:

  • AWS Sagemaker Orchestrator: Added full support for running dynamic pipelines with seamless transition from existing settings and faster execution through direct use of training jobs. PR #4232

  • Vertex AI Orchestrator: Dynamic pipelines are now fully supported on Google Cloud's Vertex AI platform. PR #4246

  • Kubernetes Orchestrator: Improved dynamic pipeline handling by eliminating unnecessary pod restarts. PR #4261

  • Snapshot Execution: For Pro users, the new release enabled running snapshots of dynamic pipelines from the server with support for specifying pipeline parameters. PR #4253

Improved
  • Enhanced step.map(...) and step.product(...) to return a single future object instead of a list of futures, simplifying the API for step invocations. PR #4261

  • Improved placeholder run handling to prevent potential issues in dynamic pipeline execution. PR #4261

  • Added better typing for Docker build options with a new class to help with conversions between SDK and CLI. PR #4262

GCP Image Builder Regional Support

Added regional location support to the GCP Image Builder, allowing you to specify Cloud Build regions for improved performance and compliance:

  • Optional location parameter for specifying Cloud Build region

  • Uses regional Cloud Build endpoint ({location}-cloudbuild.googleapis.com) when location is set

  • Maintains backward compatibility with global endpoint as default

  • Includes input validation for location parameter

PR #4268

Integration Updates

  • Evidently Integration: Updated to version >=0.5.0 to support NumPy 2.0, resolving compatibility issues when installing packages requiring NumPy 2.0+ alongside ZenML. PR #4243


0.91.2 (2025-11-19)

See what's new and improved in version 0.91.2.

ZenML 0.91.2

Kubernetes Deployer

  • Deploy your pipelines directly on Kubernetes

  • Full integration with Kubernetes orchestrator

Learn more | PR #4127

MLflow 3.0 Support

  • Added support for the latest MLflow version

  • Improved compatibility with modern MLflow features

PR #4160

S3 Artifact Store Fixes

  • Fixed compatibility with custom S3 backends

  • Improved SSL certificate handling for RestZenStore

  • Enhanced Weights & Biases experiment tracker reliability

UI Updates

  • Remove Video Modal (#943)

  • Update Dependencies (CVE) (#945)

  • Adjust text-color (#947)

  • Sanitize Dockerfile (#948)

Fixed
  • S3 artifact store now works with custom backends (#4186)

  • SSL certificate passing for RestZenStore (#4188)

  • Weights & Biases tag length limitations (#4189)


0.91.1 (2025-11-11)

See what's new and improved in version 0.91.1.

ZenML 0.91.1

Hugging Face Deployer

  • Deploy pipelines directly to Hugging Face Spaces

  • Seamless integration with Hugging Face infrastructure

Learn more | PR #4119

Dynamic Pipelines (Experimental)

  • Introduced v1 of dynamic pipelines

  • Early feedback welcome for this experimental feature

Read the documentation | PR #4074

Kubernetes Orchestrator Enhancements

  • Container security context configuration

  • Skip owner references option

  • Improved deployment reliability

UI Updates

  • Display Deployment in Run Detail (#919)

  • Announcements Widget (#926)

  • Add Resize Observer to HTML Viz (#928)

  • Adjust Overview Pipelines (#914)

  • Fix Panel background (#882)

  • Input Styling (#911)

  • Display Schedules (#879)

Improved
  • Enhanced Kubernetes orchestrator with container security context options (#4142)

  • Better handling of owner references in Kubernetes deployments (#4146)

  • Expanded HashiCorp Vault secret store authentication methods (#4110)

  • Support for newer Databricks versions (#4144)

Fixed
  • Port reuse for local deployments

  • Parallel deployment invocations

  • Keyboard interrupt handling during monitoring

  • Case-sensitivity issues when updating entity names (#4140)


0.91.0 (2025-10-25)

See what's new and improved in version 0.91.0.

ZenML 0.91.0

Local Deployer

  • Deploy pipelines locally with full control

  • Perfect for development and testing workflows

Learn more | PR #4085

Advanced Caching System

  • File and object-based cache invalidation

  • Cache expiration for bounded lifetime

  • Custom cache functions for advanced logic

Read the documentation | PR #4040

Deployment Visualizations

  • Attach custom visualizations to deployments

  • Fully customizable deployment server settings

  • Enhanced deployment management

PR #4016 | PR #4064

Python 3.13 Support

  • Full compatibility with Python 3.13

  • MLX array materializer for Apple Silicon

PR #4053 | PR #4027

UI Updates

  • Deployment Playground: Easier to invoke and test deployments (#861)

  • Global Lists: Centralized access for deployments (#851) and snapshots (#854)

  • Create Snapshots: Create snapshots directly from the UI (#856)

  • GitHub-Flavored Markdown support (#876)

  • Resizable Panels (#873)

Improved
  • Customizable image tags for Docker builds (#4025)

  • Enhanced deployment server configuration (#4064)

  • Better integration with MLX arrays (#4027)

Fixed
  • Print capturing incompatibility with numba (#4060)

  • Hashicorp Vault secrets store mount point configuration (#4088)

Breaking Changes

  • Dropped Python 3.9 support - upgrade to Python 3.10+ (#4053)


0.90.0 (2025-10-02)

See what's new and improved in version 0.90.0.

ZenML 0.90.0

Pipeline Snapshots & Deployments

  • Capture immutable snapshots of pipeline code and configuration

  • Deploy pipelines as HTTP endpoints for online inference

  • Docker, AWS, and GCP deployer implementations

PR #3856 | PR #3920

Runtime Environment Variables

  • Configure environment variables when running pipelines

  • Support for ZenML secrets in runtime configuration

PR #3336

Dependency Management Improvements

  • Reduced base package dependencies

  • Local database dependencies moved to zenml[local] extra

  • JAX array materializer support

PR #3916 | PR #3712

UI Updates

  • Pipeline Snapshots & Deployments: Track entities introduced in ZenML 0.90.0 (#814)

Improved
  • Slimmer base package for faster installations (#3916)

  • Better dependency management

  • Enhanced JAX integration (#3712)

Breaking Changes

  • Client-Server compatibility: Must upgrade both simultaneously

  • Run templates need to be recreated

  • Base package no longer includes local database dependencies - install zenml[local] if needed (#3916)


0.85.0 (2025-09-12)

See what's new and improved in version 0.85.0.

ZenML 0.85.0

Pipeline Execution Modes

  • Flexible failure handling configuration

  • Control what happens when steps fail

  • Better pipeline resilience

Read the documentation | PR #3874

Value-Based Caching

  • Cache artifacts based on content/value, not just ID

  • More intelligent cache reuse

  • Cache policies for granular control

PR #3900

Airflow 3.0 Support

  • Full compatibility with Apache Airflow 3.0

  • Access to latest Airflow features and improvements

PR #3922

UI Updates

  • Timeline View: New way to visualize pipeline runs alongside the DAG (#799)

  • Client-Side Structured Logs (#801)

  • Default Value for Arrays (#798)

Improved
  • Enhanced caching system with value-based caching (#3900)

  • More granular cache policy control

  • Better pipeline execution control (#3874)

Breaking Changes

  • Local orchestrator now continues execution after step failures

  • Docker package installer default switched from pip to uv (#3935)

  • Log endpoint format changed (#3845)


0.84.3 (2025-08-27)

See what's new and improved in version 0.84.3.

ZenML 0.84.3

ZenML Pro Service Account Authentication

  • CLI login support via zenml login --api-key

  • Service account API keys for programmatic access

  • Organization-level access for automated workflows

PR #3895 | PR #3908

ZenML Pro Service Account Authentication

  • CLI login support via zenml login --api-key

  • Service account API keys for programmatic access

  • Organization-level access for automated workflows

PR #3895 | PR #3908

Improved
  • Enhanced Kubernetes resource name sanitization (#3887)

  • Relaxed Click dependency version constraints (#3905)


0.84.2 (2025-08-06)

See what's new and improved in version 0.84.2.

ZenML 0.84.2

Kubernetes Orchestrator Improvements

  • Complete rework using Jobs instead of raw pods

  • Better robustness and automatic restarts

  • Significantly faster pipeline compilation

PR #3869 | PR #3873

Kubernetes Orchestrator Improvements

  • Complete rework using Jobs instead of raw pods

  • Better robustness and automatic restarts

  • Significantly faster pipeline compilation

PR #3869 | PR #3873

Improved
  • Enhanced Kubernetes orchestrator robustness (#3869)

  • Faster pipeline compilation for large pipelines (#3873)

  • Better logging performance (#3872)


0.84.1 (2025-07-30)

See what's new and improved in version 0.84.1.

ZenML 0.84.1

Step Exception Handling

  • Improved collection of exception information

  • Better debugging capabilities

PR #3838

External Service Accounts

  • Added support for external service accounts

  • Improved flexibility

PR #3793

Kubernetes Orchestrator Enhancements

  • Schedule management capabilities

  • Better error handling

  • Enhanced pod monitoring

PR #3847

Dynamic Fan-out/Fan-in

  • Support for dynamic patterns with run templates

  • More flexible pipeline architectures

PR #3826

Step Exception Handling

  • Improved collection of exception information

  • Better debugging capabilities

PR #3838

External Service Accounts

  • Added support for external service accounts

  • Improved flexibility

PR #3793

Kubernetes Orchestrator Enhancements

  • Schedule management capabilities

  • Better error handling

  • Enhanced pod monitoring

PR #3847

Dynamic Fan-out/Fan-in

  • Support for dynamic patterns with run templates

  • More flexible pipeline architectures

PR #3826

Fixed
  • Vertex step operator credential refresh (#3853)

  • Logging race conditions (#3855)

  • Kubernetes secret cleanup when orchestrator pods fail (#3846)


0.84.0 (2025-07-11)

See what's new and improved in version 0.84.0.

ZenML 0.84.0

Early Pipeline Stopping

  • Stop pipelines early with Kubernetes orchestrator

  • Better resource management

PR #3716

Step Retries

  • Configurable step retry mechanisms

  • Improved pipeline resilience

PR #3789

Step Status Refresh

  • Real-time status monitoring

  • Enhanced step status refresh capabilities

PR #3735

Performance Improvements

  • Thread-safe RestZenStore operations

  • Server-side processing improvements

  • Enhanced pipeline/step run fetching

PR #3758 | PR #3762 | PR #3776

UI Updates

  • Refactor Onboarding (#772) & Survey (#770)

  • Stop Runs directly from UI (#755)

  • Step Refresh (#773)

  • Support multiple log origins (#769)

Improved
  • New ZenML login experience (#3790)

  • Enhanced Kubernetes orchestrator pod caching (#3719)

  • Easier step operator/experiment tracker configuration (#3774)

  • Orchestrator pod logs access (#3778)

Fixed
  • Fixed model version fetching by UUID (#3777)

  • Visualization handling improvements (#3769)

  • Fixed data artifact fetching (#3811)

  • Path and Docker tag sanitization (#3816 | #3820)

Breaking Changes

  • Kubernetes Orchestrator Compatibility: Client and orchestrator pod versions must match exactly


Last updated

Was this helpful?