🐍Python Client
Interacting with your ZenML instance through the ZenML Client.
Pipelines, runs, stacks, and many other ZenML resources are stored and versioned in a database within your ZenML instance behind the scenes. The ZenML Python Client
allows you to fetch, update, or even create any of these resources programmatically in Python.
In all other programming languages and environments, you can interact with ZenML resources through the REST API endpoints of your ZenML server instead. Checkout the /docs/
page of your server for an overview of all available endpoints.
Usage Example
The following example shows how to use the ZenML Client to fetch the last 10 pipeline runs that you ran yourself on the stack that you have currently set:
List of Resources
These are the main ZenML resources that you can interact with via the ZenML Client:
Pipelines, Runs, Artifacts
Pipelines: The pipelines that were implicitly tracked when running ZenML pipelines.
Pipeline Runs: Information about all pipeline runs that were executed on your ZenML instance.
Run Templates: Templates to run pipelines from the server or dashboard.
Step Runs: The steps of all pipeline runs. Mainly useful for directly fetching a specific step of a run by its ID.
Artifacts: Information about all artifacts that were written to your artifact stores as part of pipeline runs.
Schedules: Metadata about the schedules that you have used to schedule pipeline runs.
Builds: The pipeline-specific Docker images that were created when containerizing your pipeline.
Code Repositories: The git code repositories that you have connected with your ZenML instance. See here for more information.
Checkout the documentation on fetching runs for more information on the various ways how you can fetch and use the pipeline, pipeline run, step run, and artifact resources in code.
Stacks, Infrastructure, Authentication
Stack: The stacks registered in your ZenML instance.
Stack Components: The stack components registered in your ZenML instance, e.g., all orchestrators, artifact stores, model deployers, ...
Flavors: The stack component flavors available to you, including:
Built-in flavors like the local orchestrator,
Integration-enabled flavors like the Kubeflow orchestrator,
Custom flavors that you have created yourself.
User: The users registered in your ZenML instance. If you are running locally, there will only be a single
default
user.Secrets: The infrastructure authentication secrets that you have registered in the ZenML Secret Store.
Service Connectors: The service connectors that you have set up to connect ZenML to your infrastructure.
Client Methods
Reading and Writing Resources
List Methods
Get a list of resources, e.g.:
These methods always return a Page of resources, which behaves like a standard Python list and contains, by default, the first 50 results. You can modify the page size by passing the size
argument or fetch a subsequent page by passing the page
argument to the list method.
You can further restrict your search by passing additional arguments that will be used to filter the results. E.g., most resources have a user_id
associated with them that can be set to only list resources created by that specific user. The available filter argument options are different for each list method; check out the method declaration in the Client SDK documentation to find out which exact arguments are supported or have a look at the fields of the corresponding filter model class.
Except for pipeline runs, all other resources will by default be ordered by creation time ascending. E.g., client.list_artifacts()
would return the first 50 artifacts ever created. You can change the ordering by specifying the sort_by
argument when calling list methods.
Get Methods
Fetch a specific instance of a resource by either resource ID, name, or name prefix, e.g.:
Create, Update, and Delete Methods
Methods for creating / updating / deleting resources are only available for some of the resources and the required arguments are different for each resource. Checkout the Client SDK Documentation to find out whether a specific resource supports write operations through the Client and which arguments are required.
Active User and Active Stack
For some use cases you might need to know information about the user that you are authenticated as or the stack that you have currently set as active. You can fetch this information via the client.active_user
and client.active_stack_model
properties respectively, e.g.:
Resource Models
The methods of the ZenML Client all return Response Models, which are Pydantic Models that allow ZenML to validate that the returned data always has the correct attributes and types. E.g., the client.list_pipeline_runs
method always returns type Page[PipelineRunResponseModel]
.
You can think of these models as similar to types in strictly-typed languages, or as the requirements of a single endpoint in an API. In particular, they are not related to machine learning models like decision trees, neural networks, etc.
ZenML also has similar models that define which information is required to create, update, or search resources, named Request Models, Update Models, and Filter Models respectively. However, these models are only used for the server API endpoints, and not for the Client methods.
To find out which fields a specific resource model contains, checkout the ZenML Models SDK Documentation and expand the source code to see a list of all fields of the respective model. Note that all resources have Base Models that define fields that response, request, update, and filter models have in common, so you need to take a look at the base model source code as well.
Last updated