Controlling Model versions

Each model can have many versions. Model versions are a way for you to track different iterations of your training process, complete with some extra dashboard and API functionality to support the full ML lifecycle.

E.g. Based on your business rules during training, you can associate model version with stages and promote them to production. You have an interface that allows you to link these versions with non-technical artifacts and data, e.g. business data, datasets, or even stages in your process and workflow.

Model versions are created implicitly as you are running your machine learning training, so you don't have to immediately think about this. If you want more control over versions, our API has you covered, with an option to explicitly name your versions.

Explicitly name your model version

If you want to explicitly name your model version, you can do so by passing in the version argument to the Model object. If you don't do this, ZenML will automatically generate a version number for you.

from zenml import Model, step, pipeline

model= Model(
    name="my_model",
    version="1.0.5"
)

# The step configuration will take precedence over the pipeline
@step(model=model)
def svc_trainer(...) -> ...:
    ...

# This configures it for all steps within the pipeline
@pipeline(model=model)
def training_pipeline( ... ):
    # training happens here

Here we are specifically setting the model configuration for a particular step or for the pipeline as a whole.

Please note in the above example if the model version exists, it is automatically associated with the pipeline and becomes active in the pipeline context. Therefore, a user should be careful and intentional as to whether you want to create a new pipeline, or fetch an existing one. See below for an example of fetching a model from an existing version/stage.

Use name templates for your model versions

If you want to continuously run the same project, but keep track of your model versions using semantical naming, you can rely on templated naming in the version argument to the Model object. Instead of static model version name from the previous section, templated names will be unique with every new run, but also will be semantically searchable and readable by your team.

from zenml import Model, step, pipeline

model= Model(
    name="my_model",
    version="experiment_with_phi_3_{date}_{time}"
)

# The step configuration will take precedence over the pipeline
@step(model=model)
def llm_trainer(...) -> ...:
    ...

# This configures it for all steps within the pipeline
@pipeline(model=model)
def training_pipeline( ... ):
    # training happens here

Here we are specifically setting the model configuration for a particular step or for the pipeline as a whole. Once you run this pipeline it will produce a model version with a name evaluated at a runtime, like experiment_with_phi_3_2024_08_30_12_42_53. Subsequent runs will also have unique but readable names.

We currently support following placeholders to be used in model version name templates:

• {date}: current date

• {time}: current time in UTC format

Fetching model versions by stage

A common pattern is to assign a special stage to a model version, i.e. production, staging, development etc. This marks this version especially, and can be used to fetch it using a particular semantic meaning, disconnected from the concrete model version. A model version can be assigned a particular stage in the dashboard or by executing the following command in the CLI:

zenml model version update MODEL_NAME --stage=STAGE

These stages can then be passed in as a version to fetch the right model version at a later point:

from zenml import Model, step, pipeline

model= Model(
    name="my_model",
    version="production"
)

# The step configuration will take precedence over the pipeline
@step(model=model)
def svc_trainer(...) -> ...:
    ...

# This configures it for all steps within the pipeline
@pipeline(model=model)
def training_pipeline( ... ):
    # training happens here

Autonumbering of versions

ZenML automatically numbers your model versions for you. If you don't specify a version number, or if you pass None into the version argument of the Model object, ZenML will automatically generate a version number (or a new version, if you already have a version) for you. For example if we had a model version really_good_version for model my_model and we wanted to create a new version of this model, we could do so as follows:

from zenml import Model, step

model = Model(
    name="my_model",
    version="even_better_version"
)

@step(model=model)
def svc_trainer(...) -> ...:
    ...

A new model version will be created and ZenML will track that this is the next in the iteration sequence of the models using the number property. If really_good_version was the 5th version of my_model, then even_better_version will be the 6th version of my_model.

from zenml import Model

earlier_version = Model(
    name="my_model",
    version="really_good_version"
).number # == 5

updated_version = Model(
    name="my_model",
    version="even_better_version"
).number # == 6