Continuous Training and Deployment

As an organization develops across the MLOps maturity model , the terms Continuous Training and Continuous Deployment (CT/CD) get more relevant.
Continuous Training refers to the paradigm where a team deploys training pipelines that run automatically to train models on new (fresh) data. (e.g. every two weeks, take the latest data from an API, and train a new model on it.)
Continuous Deployment refers to the paradigm where newly trained models are automatically deployed to a prediction service/server, when a criterion in production is fulfilled (e.g. if a trained model has a certain accuracy, or overall performs better than the previous one, deploy it in production.)
ZenML allows both paradigms with Schedules , Model Deployers and Services.
Setting a pipeline schedule for Continuous Training (CT)
ZenML supports running pipelines on a schedule as follows:
1
import datetime
2
from datetime import timedelta
3
from zenml.pipelines import Schedule
4
​
5
# Create a schedule to run a pipeline every minute for the next ten minutes
6
schedule = Schedule(
7
    start_time=datetime.now(),
8
    end_time=datetime.now() + timedelta(minutes=10),
9
    interval_second=60,
10
)
11
​
12
# Run the pipeline on a schedule
13
my_pipeline(
14
    ...  # steps
15
).run(schedule=schedule)
Copied!
The above deploys a pipeline, potentially on a production stack, on a defined schedule. If the pipeline has a well-defined data loading/importing step, then one can deploy it to run and train new models on fresh data on a regular basis. This enables Continuous Training with ZenML.
Interacting with services for Continuous Deployment (CD)
Continuous Training is necessary in a mature MLOps setting, but Continuous Deployment completes the picture. Continuous Deployment is also interesting because it involves interacting with systems that are longer-lived than a pipeline run.
ZenML interacts with such external systems (e.g. like prediction servers) through the Model Deployer abstraction. The concrete implementation of this abstraction deals with functionality concerning the life-cycle management and tracking of external model deployment servers (e.g. processes, containers, Kubernetes deployments etc.), which are represented in ZenML using another abstraction: Services.
The first thing needed to be able to deploy machine learning models to external model serving platforms with ZenML in a continuous deployment manner is to have a Model Deployer registered as part of your ZenML Stack. MLflow and Seldon Core are two examples of Model Deployers already provided by ZenML as an integration, with many other Model Deployers to follow. The Model Deployer abstraction is also meant to be easily extensible by anyone who wishes to implement their own flavor and integrate ZenML with their model serving tool of choice.
There are three major roles that a Model Deployer plays in a ZenML Stack:
1.it holds all the stack related configuration attributes required to interact with the remote model serving tool, service or platform (e.g. hostnames, URLs, references to credentials, other client related configuration parameters). The following are examples of configuring the MLflow and Seldon Core Model Deployers and registering them as a Stack component:
1
zenml integration install mlflow
2
zenml model-deployer register mlflow --flavor=mlflow
3
zenml stack register local_with_mlflow -m default -a default -o default -d mlflow --set
Copied!
1
zenml integration install seldon
2
zenml model-deployer register seldon --flavor=seldon \
3
--kubernetes_context=zenml-eks --kubernetes_namespace=zenml-workloads \
4
--base_url=http://abb84c444c7804aa98fc8c097896479d-377673393.us-east-1.elb.amazonaws.com
5
...
6
zenml stack register seldon_stack -m default -a aws -o default -d seldon
Copied!
2.it implements the continuous deployment logic necessary to deploy models in a way that updates an existing model server that is already serving a previous version of the same model instead of creating a new model server for every new model version. Every model server that the Model Deployer provisions externally to deploy a model is represented internally as a Service object that may be accessed for visibility and control over a single model deployment. This functionality can be consumed directly from ZenML pipeline steps, but it can also be used outside of the pipeline to deploy ad-hoc models. The following code is an example of using the Seldon Core Model Deployer to deploy a model inside a ZenML pipeline step:
1
from zenml.artifacts import ModelArtifact
2
from zenml.environment import Environment
3
from zenml.integrations.seldon.model_deployers import SeldonModelDeployer
4
from zenml.integrations.seldon.services.seldon_deployment import (
5
  SeldonDeploymentConfig,
6
  SeldonDeploymentService,
7
)
8
from zenml.steps import (
9
  STEP_ENVIRONMENT_NAME,
10
  StepContext,
11
  step,
12
)
13
​
14
@step(enable_cache=True)
15
def seldon_model_deployer_step(
16
  context: StepContext,
17
  model: ModelArtifact,
18
) -> SeldonDeploymentService:
19
  model_deployer = SeldonModelDeployer.get_active_model_deployer()
20
​
21
  # get pipeline name, step name and run id
22
  step_env = Environment()[STEP_ENVIRONMENT_NAME]
23
​
24
  service_config=SeldonDeploymentConfig(
25
      model_uri=model.uri,
26
      model_name="my-model",
27
      replicas=1,
28
      implementation="TENSORFLOW_SERVER",
29
      secret_name="seldon-secret",
30
      pipeline_name = step_env.pipeline_name,
31
      pipeline_run_id = step_env.pipeline_run_id,
32
      pipeline_step_name = step_env.step_name,
33
  )
34
​
35
  service = model_deployer.deploy_model(
36
      service_config, replace=True, timeout=300
37
  )
38
​
39
  print(
40
      f"Seldon deployment service started and reachable at:\n"
41
      f"    {service.prediction_url}\n"
42
  )
43
​
44
  return service
Copied!
3.the Model Deployer acts as a registry for all Services that represent remote model servers. External model deployment servers can be listed and filtered using a variety of criteria, such as the name of the model or the names of the pipeline and step that was used to deploy the model. The Service objects returned by the Model Deployer can be used to interact with the remote model server, e.g. to get the operational status of a model server, the prediction URI that it exposes, or to stop or delete a model server:
1
from zenml.integrations.seldon.model_deployers import SeldonModelDeployer
2
​
3
model_deployer = SeldonModelDeployer.get_active_model_deployer()
4
services = model_deployer.find_model_server(
5
    pipeline_name="continuous-deployment-pipeline",
6
    pipeline_step_name="seldon_model_deployer_step",
7
    model_name="my-model",
8
)
9
if services:
10
    if services[0].is_running:
11
        print(
12
            f"Seldon deployment service started and reachable at:\n"
13
            f"    {services[0].prediction_url}\n"
14
        )
15
    elif services[0].is_failed:
16
        print(
17
            f"Seldon deployment service is in a failure state. "
18
            f"The last error message was: {services[0].status.last_error}"
19
        )
20
    else:
21
        print(f"Seldon deployment service is not running")
22
​
23
        # start the service
24
        services[0].start(timeout=100)
25
​
26
    # delete the service
27
    model_deployer.delete_service(services[0].uuid, timeout=100, force=False)
Copied!
When a Model Deployer is part of the active ZenML Stack, it is also possible to interact with it from the CLI to list, start, stop or delete the model servers that is manages:
1
$ zenml served-models list
2
┏━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
3
┃ STATUS │ UUID                                 │ PIPELINE_NAME                  │ PIPELINE_STEP_NAME         ┃
4
┠────────┼──────────────────────────────────────┼────────────────────────────────┼────────────────────────────┨
5
┃   ✅   │ 8cbe671b-9fce-4394-a051-68e001f92765 │ continuous_deployment_pipeline │ seldon_model_deployer_step ┃
6
┗━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
7
​
8
$ zenml served-models describe 8cbe671b-9fce-4394-a051-68e001f92765
9
                          Properties of Served Model 8cbe671b-9fce-4394-a051-68e001f92765                          
10
┏━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
11
┃ MODEL SERVICE PROPERTY │ VALUE                                                                                  ┃
12
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
13
┃ MODEL_NAME             │ mnist                                                                                  ┃
14
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
15
┃ MODEL_URI              │ s3://zenfiles/seldon_model_deployer_step/output/884/seldon                             ┃
16
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
17
┃ PIPELINE_NAME          │ continuous_deployment_pipeline                                                         ┃
18
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
19
┃ PIPELINE_RUN_ID        │ continuous_deployment_pipeline-11_Apr_22-09_39_27_648527                               ┃
20
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
21
┃ PIPELINE_STEP_NAME     │ seldon_model_deployer_step                                                             ┃
22
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
23
┃ PREDICTION_URL         │ http://abb84c444c7804aa98fc8c097896479d-377673393.us-east-1.elb.amazonaws.com/seldon/… ┃
24
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
25
┃ SELDON_DEPLOYMENT      │ zenml-8cbe671b-9fce-4394-a051-68e001f92765                                             ┃
26
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
27
┃ STATUS                 │ ✅                                                                                     ┃
28
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
29
┃ STATUS_MESSAGE         │ Seldon Core deployment 'zenml-8cbe671b-9fce-4394-a051-68e001f92765' is available       ┃
30
┠────────────────────────┼────────────────────────────────────────────────────────────────────────────────────────┨
31
┃ UUID                   │ 8cbe671b-9fce-4394-a051-68e001f92765                                                   ┃
32
┗━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
33
​
34
$ zenml served-models get-url 8cbe671b-9fce-4394-a051-68e001f92765
35
  Prediction URL of Served Model 8cbe671b-9fce-4394-a051-68e001f92765 is:
36
  http://abb84c444c7804aa98fc8c097896479d-377673393.us-east-1.elb.amazonaws.com/seldon/zenml-workloads/zenml-8cbe67
37
1b-9fce-4394-a051-68e001f92765/api/v0.1/predictions
38
​
39
$ zenml served-models delete 8cbe671b-9fce-4394-a051-68e001f92765
Copied!
Services can be passed through steps like any other object, and used to interact with the external systems that they represent:
1
from zenml.steps import step
2
​
3
@step
4
def my_step(my_service: MyService) -> ...:
5
    if not my_service.is_running:
6
        my_service.start()  # starts service
7
    my_service.stop()  # stops service
Copied!
The ZenML integrations that provide Model Deployer stack components also include standard pipeline steps that can simply be inserted into any pipeline to achieve a continuous model deployment workflow. These steps take care of all the aspects of continuously deploying models to an external server and saving the Service configuration into the Artifact Store, where they can be loaded at a later time and re-create the initial conditions used to serve a particular model.
You can see concrete examples of using a Model Deployer and model deployment builtin steps to implement a continuous training and continuous deployment pipeline with the MLflow deployment and the Seldon Core deployment examples.
Following are some code snippets extracted from the above mentioned examples that give a quick glimpse into using the MLflow and Seldon Core standard model deployer steps:
1
from zenml.integrations.constants import MLFLOW, TENSORFLOW
2
from zenml.pipelines import pipeline
3
from zenml.integrations.mlflow.steps import (
4
    MLFlowDeployerConfig,
5
    mlflow_model_deployer_step,
6
)
7
​
8
...
9
​
10
​
11
@pipeline(required_integrations=[MLFLOW, TENSORFLOW])
12
def continuous_deployment_pipeline(
13
        importer,
14
        normalizer,
15
        trainer,
16
        evaluator,
17
        deployment_trigger,
18
        model_deployer,
19
):
20
    x_train, y_train, x_test, y_test = importer()
21
    x_trained_normed, x_test_normed = normalizer(x_train=x_train, x_test=x_test)
22
    model = trainer(x_train=x_trained_normed, y_train=y_train)
23
    accuracy = evaluator(x_test=x_test_normed, y_test=y_test, model=model)
24
    deployment_decision = deployment_trigger(accuracy=accuracy)
25
    model_deployer(deployment_decision, model)
26
​
27
​
28
...
29
​
30
# Initialize a continuous deployment pipeline
31
deployment = continuous_deployment_pipeline(
32
    ...,
33
    model_deployer = mlflow_model_deployer_step(
34
        config=MLFlowDeployerConfig(model_name="model", workers=3, timeout=20))
35
)
36
​
37
...
38
​
39
deployment.run()
Copied!
1
from zenml.integrations.constants import SELDON, TENSORFLOW, SKLEARN
2
from zenml.pipelines import pipeline
3
from zenml.integrations.seldon.services import SeldonDeploymentConfig
4
from zenml.integrations.seldon.steps import (
5
    SeldonDeployerStepConfig,
6
    seldon_model_deployer_step,
7
)
8
​
9
...
10
​
11
​
12
@pipeline(required_integrations=[SELDON, TENSORFLOW, SKLEARN])
13
def continuous_deployment_pipeline(
14
    importer,
15
    normalizer,
16
    trainer,
17
    evaluator,
18
    deployment_trigger,
19
    model_deployer,
20
):
21
    # Link all the steps artifacts together
22
    x_train, y_train, x_test, y_test = importer()
23
    x_trained_normed, x_test_normed = normalizer(x_train=x_train, x_test=x_test)
24
    model = trainer(x_train=x_trained_normed, y_train=y_train)
25
    accuracy = evaluator(x_test=x_test_normed, y_test=y_test, model=model)
26
    deployment_decision = deployment_trigger(accuracy=accuracy)
27
    model_deployer(deployment_decision, model)
28
​
29
​
30
...
31
​
32
# Initialize a continuous deployment pipeline run
33
deployment = continuous_deployment_pipeline(
34
    ...,
35
    model_deployer = seldon_model_deployer_step(
36
        config=SeldonDeployerStepConfig(
37
            service_config=SeldonDeploymentConfig(
38
                model_name="model",
39
                replicas=1,
40
                implementation="SKLEARN_SERVER",
41
                secret_name="seldon-init-container-secret",
42
            ),
43
            timeout=120,
44
        ))
45
)
46
​
47
deployment.run()
Copied!
The model deployment Service can also be used to interact with a model prediction server with the following interface:
1
# sends data to prediction service with a unified interface
2
my_deployment_service.predict(my_data)
Copied!
Advanced Guide - Previous
Execute Pipelines in the Cloud
Next - Advanced Guide
Manage your Secrets
Last modified 10d ago
Export as PDF
Copy link
Edit on GitHub
Contents
Setting a pipeline schedule for Continuous Training (CT)
Interacting with services for Continuous Deployment (CD)