Integrations

Use these tools out-of-the-box with ZenML.

This is an older version of the ZenML documentation. To check the latest version please visit https://docs.zenml.io

Integrations

ZenML integrates with many different third-party tools as implementations for many different ZenML abstractions.

Once code is organized into a ZenML pipeline, you can supercharge your ML development with powerful integrations on multiple MLOps stacks. There are lots of moving parts for all the MLOps tooling and infrastructure you require for ML in production and ZenML aims to bring it all together under one roof.

For example, we currently support Airflow and Kubeflow as third-party orchestrators for your ML pipeline code. Experiment trackers like MLflow Tracking and Weights & Biases can easily be added to your ZenML pipeline. And you can seamlessly transition from a local MLflow deployment to a deployed model on Kubernetes using Seldon Core.

All of this allows you to write your code now and add the right tool for the job as soon as the need arises.

ZenML is the glue

These are the third-party integrations that ZenML currently supports:

Integration	Status	Type	Implementation Notes	Example
Apache Airflow	✅	Orchestrator	Works for local environment.	airflow_orchestration
Apache Beam	✅	Distributed Processing
AWS	✅	Container Registry	Use the AWS container registry to store your containers.
AWS	✅	Secrets Manager	Use AWS as a secrets manager.	cloud_secrets_manager
AWS	✅	Step Operator	Sagemaker as a ZenML step operator.	sagemaker_step_operator
Azure	✅	Artifact Store	Use Azure Blob Storage buckets as ZenML artifact stores.
Azure	✅	Step Operator	Use AzureML as a step operator to supercharge specific steps.	azureml_step_operator
BentoML	⛏	Deployment	Looking for community implementors.
Dash	✅	Visualizer	For Pipeline and PipelineRun visualization objects.
Evidently	✅	Monitoring	Allows for visualization of drift as well as export of a Profile object.	drift_detection
Facets	✅	Visualizer	Quickly visualize your datasets using facets.	facets_visualize_statistics
Feast	✅	Feature Store	Use Feast with Redis for your online features.	feast_feature_store
GitHub	✅	Orchestrator	Use GitHub Actions to orchestrate your ZenML pipelines.	github_actions_orchestration
GCP	✅	Artifact Store	Use GCS buckets as a ZenML artifact store.
GCP	✅	Step Secrets Manager	Use the GCP Secret Manager.	cloud_secrets_manager
GCP	✅	Step Operator	Vertex AI as a ZenML step operator.	vertex_step_operator
GCP	✅	Orchestrator	Execute your ZenML pipelines using Vertex AI Pipelines.
Graphviz	✅	Visualizer	For Pipeline and PipelineRun visualization objects.
Great Expectations	⛏	Data Validation	Looking for community implementors.
Hugging Face	✅	Materializer	Use Hugging Face tokenizers, datasets and models.	huggingface
KServe	⛏	Inference	Looking for community implementors.
Kubeflow	✅	Orchestrator	Either full Kubeflow or Kubeflow Pipelines. Works for local environments currently.	kubeflow_pipelines_orchestration
Kubernetes	✅	Orchestrator	Only works with remote clusters currently.	kubernetes_orchestration
lightgbm	✅	Training	Support for Booster and Dataset materialization.	lightgbm
MLflow Tracking	✅	Experiment Tracking	Tracks your pipelines and your training runs.	mlflow
MLflow Deployment	✅	Deployment	Deploys models with the MLflow scoring server.	mlflow
NeuralProphet	✅	Training	Enables materializing NeuralProphet models.	neural_prophet
numpy	✅	Exploration
pandas	✅	Exploration
Plotly	✅	Visualizer	For Pipeline and PipelineRun visualization objects.
PyTorch	✅	Training		pytorch
PyTorch Lightning	✅	Training
S3	✅	Artifact Store	Use S3 buckets as ZenML artifact stores.
scikit-learn	✅	Training
scipy	✅	Materializer	Use sparse matrices.
Seldon Core	✅	Deployment	Seldon Core as a model deployer.	seldon_deployment
Slack	✅	Alerter	Send automated alerts to Slack.	slack_alert
Tensorflow	✅	Training, Visualizer	TensorBoard support.	kubeflow_pipelines_orchestration
Weights & Biases	✅	Experiment Tracking	Tracks your pipelines and your training runs.	wandb_tracking
whylogs	✅	Logging	Integration fully implemented for data logging.	whylogs_data_profiling
xgboost	✅	Training	Support for Booster and DMatrix materialization.	xgboost
Vault	✅	Secrets Manager	Use Vault Key/Value Secrets Engine

✅ means the integration is already implemented. ⛏ means we are looking to implement the integration soon.

Help us with integrations!

There are many tools in the ML / MLOps field. We have made an initial prioritization of which tools to support with integrations, but we also welcome community contributions. Check our Contributing Guide for more details on how best to contribute.