Integrations

ZenML integrates with many different third-party tools as implementations for many different ZenML abstractions.
Once code is organized into a ZenML pipeline, you can supercharge your ML development with powerful integrations on multiple MLOps stacks. There are lots of moving parts for all the MLOps tooling and infrastructure you require for ML in production and ZenML aims to bring it all together under one roof.
For example, we currently support Airflow and Kubeflow as third-party orchestrators for your ML pipeline code. Experiment trackers like MLflow Tracking and Weights & Biases can easily be added to your ZenML pipeline. And you can seamlessly transition from a local MLflow deployment to a deployed model on Kubernetes using Seldon Core.
All of this allows you to write your code now and add the right tool for the job as soon as the need arises.
ZenML is the glue
These are the third-party integrations that ZenML currently supports:
Integration
Status
Type
Implementation Notes
Example
Apache Airflow
✅
Orchestrator
Works for local environment.
​airflow_orchestration​
Apache Beam
✅
Distributed Processing
​
​
AWS
✅
Container Registry
Use the AWS container registry to store your containers.
​
AWS
✅
Secrets Manager
Use AWS as a secrets manager.
​cloud_secrets_manager​
AWS
✅
Step Operator
Sagemaker as a ZenML step operator.
​sagemaker_step_operator​
Azure
✅
Artifact Store
Use Azure Blob Storage buckets as ZenML artifact stores.
​
Azure
✅
Step Operator
Use AzureML as a step operator to supercharge specific steps.
​azureml_step_operator​
BentoML
⛏
Deployment
Looking for community implementors.
​
Dash
✅
Visualizer
For Pipeline and PipelineRun visualization objects.
​lineage​
Evidently
✅
Monitoring
Allows for visualization of drift as well as export of a Profile object.
​drift_detection​
Facets
✅
Visualizer
Quickly visualize your datasets using facets.
​statistics​
Feast
✅
Feature Store
Use Feast with Redis for your online features.
​feature_store​
GitHub
✅
Orchestrator
Use GitHub Actions to orchestrate your ZenML pipelines.
​github_actions_orchestration​
GCP
✅
Artifact Store
Use GCS buckets as a ZenML artifact store.
​
GCP
✅
Step Secrets Manager
Use the GCP Secret Manager.
​cloud_secrets_manager​
GCP
✅
Step Operator
Vertex AI as a ZenML step operator.
​vertex_step_operator​
GCP
✅
Orchestrator
Execute your ZenML pipelines using Vertex AI Pipelines.
​
Graphviz
✅
Visualizer
For Pipeline and PipelineRun visualization objects.
​dag_visualizer​
Great Expectations
⛏
Data Validation
Looking for community implementors.
​
Hugging Face
✅
Materializer
Use Hugging Face tokenizers, datasets and models.
​huggingface​
✅ means the integration is already implemented. ⛏ means we are looking to implement the integration soon.
Help us with integrations!
There are many tools in the ML / MLOps field. We have made an initial prioritization of which tools to support with integrations, but we also welcome community contributions. Check our Contributing Guide for more details on how best to contribute.

Advanced Guide - Previous

Stacks, Components, Flavors

Next - Advanced Guide

Track Metadata in the Cloud

Last modified 10d ago

Export as PDF

Copy link

Edit on GitHub

Contents

Help us with integrations!