Functional API
Build production ML pipelines from the simple step interface.
The Functional ZenML API is defined by the primitive @step and @pipeline decorators. These make it easy to quickly take your functional code into a ZenML pipeline. If you want more control over the different steps the Class based API is the way to go.
A user may also mix-and-match the Functional API with the Class Based API: All standard data types and steps that are applicable in both of these approaches.
In order to illustrate how the Functional API functions, we'll do a simple exercise to create a training pipeline from scratch (without any High Level components) and take it all the way to deployment.
Here is what we'll do in this guide:
  • Create a MNIST pipeline that trains using TensorFlow (Keras) (similar to the Quickstart).
  • Swap out implementations of the trainer and evaluator steps with scikit-learn.
  • Persist our interim data artifacts in SQL tables rather than on files.
  • Read from a dynamically changing datasource rather than a static one.
  • Deploy the pipeline on Kubeflow Pipelines.
If you just want to see all the code for each chapter of the guide, head over to the GitHub version
If not, then get your environment ready and follow along!

Set up locally

In order to run the chapters of the guide, you need to install and initialize ZenML:
1
pip install zenml
2
zenml integration install tensorflow
3
zenml integration install sklearn
4
pip install sqlalchemy
5
6
# pull example
7
zenml example pull functional_api
8
cd zenml_examples/functional_api
9
10
# initialize
11
zenml init
Copied!
In general, to run each chapter you can do:
1
python chapter_*.py # for the chapter of your choice
Copied!
Note before executing each chapter, make sure to clean the old chapter artifact and metadata store:
1
rm -rf .zen
2
zenml init # start again
Copied!

Clean up

In order to clean up, delete the remaining zenml references.
1
rm -rf zenml_examples
Copied!
Press next to start the first chapter!
Last modified 22h ago
Export as PDF
Copy link
Edit on GitHub