Post Execution Workflow

Search…

0.6.0

Post-execution workflow
After executing a pipeline, the user needs to be able to fetch it from history and perform certain tasks. This page captures these workflows at an orbital level.
Component Hierarchy
In the context of a post-execution workflow, there is an implied hierarchy of some basic ZenML components:
repository -> pipelines -> runs -> steps -> outputs
​
# where -> implies a 1-many relationship.
Repository
The highest level repository object is where to start from.
from zenml.repository import Repository
​
repo = Repository()
Pipelines
# get all pipelines from all stacks
pipelines = repo.get_pipelines()  
​
# or get one pipeline by name and/or stack key
pipeline = repo.get_pipeline(pipeline_name=..., stack_key=...)
Runs
runs = pipeline.runs  # all runs of a pipeline chronlogically ordered
run = runs[-1]  # latest run
​
# or get it by name
run = pipeline.get_run(run_name="custom_pipeline_run_name")
Steps
# at this point we switch from the `get_` paradigm to properties
steps = run.steps  # all steps of a pipeline
step = steps[0]
print(step.entrypoint_name)
Outputs
# The outputs of a step
# if multiple outputs they are accessible by name
output = step.outputs["output_name"]
​
# if one output, use the `.output` property instead 
output = step.output 
​
# will get you the value from the original materializer used in the pipeline
output.read()  
Materializing outputs (or inputs)
Once an output artifact is acquired from history, one can visualize it with any chosen Visualizer.
from zenml.materializers import PandasMaterializer
​
​
df = output.read(materializer_class=PandasMaterializer)
df.head()
Retrieving Model
from zenml.integrations.tensorflow.materializers.keras_materializer import KerasMaterializer    
​
​
model = output.read(materializer_class=KerasMaterializer)
model  # read keras.Model
Visuals
Seeing statistics
from zenml.integrations.facets.visualizers.facet_statistics_visualizer import (
    FacetStatisticsVisualizer,
)
​
FacetStatisticsVisualizer().visualize(output)
It produces the following visualization:
Statistics for boston housing dataset

Guides - Previous

General

Naming a pipeline run

Last modified 6mo ago

Export as PDF

Copy link

Edit on GitHub

Outline

Post-execution workflow

Component Hierarchy

Repository

Materializing outputs (or inputs)

Retrieving Model

Visuals

Seeing statistics