Container Registries

A container registry is a store for (Docker) containers. A ZenML workflow involving a container registry would automatically containerize your code to be transported across stacks running remotely. As part of the deployment to the cluster, the ZenML base image would be downloaded (from a cloud container registry) and used as the basis for the deployed 'run'.
For instance, when you are running a local container-based stack, you would therefore have a local container registry which stores the container images you create that bundle up your pipeline code. You could also use a remote container registry like the Elastic Container Registry at AWS in a production setting.
Before reading this chapter, make sure that you are familiar with the concept of stacks, stack components and their flavors.
Base Abstraction
In the current version of ZenML, container registries have a rather simple base abstraction. In essence, each container registry features a uri as an instance configuration and a non-abstract prepare_image_push method for validation.
1
from typing import ClassVar
2
​
3
from zenml.enums import StackComponentType
4
from zenml.stack import StackComponent
5
from zenml.utils import docker_utils
6
​
7
​
8
class BaseContainerRegistry(StackComponent):
9
    """Base class for all ZenML container registries."""
10
    
11
    # Instance configuration
12
    uri: str
13
​
14
    # Class variables
15
    TYPE: ClassVar[StackComponentType] = StackComponentType.CONTAINER_REGISTRY
16
​
17
    def prepare_image_push(self, image_name: str) -> None:
18
        """Conduct necessary checks/preparations before an image gets pushed."""
19
​
20
    def push_image(self, image_name: str) -> None:
21
        """Pushes a docker image."""
22
        if not image_name.startswith(self.uri):
23
            raise ValueError(
24
                f"Docker image `{image_name}` does not belong to container "
25
                f"registry `{self.uri}`."
26
            )
27
​
28
        self.prepare_image_push(image_name)
29
        docker_utils.push_docker_image(image_name)
Copied!
This is a slimmed-down version of the base implementation which aims to highlight the abstraction layer. In order to see the full implementation and get the complete docstrings, please check the API docs.
List of available container registry
ZenML comes built-in with several flavors for container registries:
Text
Flavor
Integration
DefaultContainerRegistry
default
built-in
DockerHubContainerRegistry
dockerhub
built-in
GCPContainerRegistry
gcp
built-in
AzureContainerRegistry
azure
built-in
AWSContainerRegistry
aws
aws
If you would like to see the available flavors for container registries, you can use the command:
1
zenml container-registry flavor list
Copied!
Building your own container registry
If you want to create your own custom flavor for a container registry, you can follow the following steps:
1.Create a class which inherits from the BaseContainerRegistry.
2.Define the FLAVOR class variable.
3.If you need to execute any checks/validation before the image gets pushed, you can define these operations in the prepare_image_push method. As an example, you can check the AWSContainerRegistry.
4.Once the prepare_image_push gets completed, the push_image method will come into play and utilize a DockerClient object to push the image.
Once you are done with the implementation, you can register it through the CLI as:
1
zenml container-registry flavor register <THE-SOURCE-PATH-OF-YOUR-REGISTRY>
Copied!

Extending ZenML - Previous

Metadata Stores

Next - Extending ZenML

Step Operators

Last modified 10d ago

Export as PDF

Copy link

Edit on GitHub

Contents

Base Abstraction

List of available container registry

Building your own container registry