AWS Image Builder
Building container images with AWS CodeBuild
Last updated
Was this helpful?
Building container images with AWS CodeBuild
Last updated
Was this helpful?
The AWS image builder is an flavor provided by the ZenML aws integration that uses to build container images.
You should use the AWS image builder if:
you're unable to install or use on your client machine.
you're already using AWS.
your stack is mainly composed of other AWS components such as the or the .
To use the AWS image builder, you need:
The ZenML aws integration installed. If you haven't done so, run:
An where the build context will be uploaded, so AWS CodeBuild can access it.
Recommended: an where the built image will be pushed. The AWS CodeBuild service can also work with other container registries, but must be enabled in this case.
An created in the AWS account and region where you want to build the Docker images, preferably in the same region as the ECR container registry where images will be pushed (if applicable). The CodeBuild project configuration is largely irrelevant, as ZenML will override most of the default settings for each build according to the . Some example default configuration values are:
Source Type: Amazon S3
Bucket: The same S3 bucket used by the ZenML S3 Artifact Store.
S3 folder: any value (e.g. codebuild);
Environment Type: Linux Container
Environment Image: bentolor/docker-dind-awscli
Privileged Mode: false
The user must take care that the Service Role attached to the CodeBuild project also has the necessary permissions to access the S3 bucket to read objects and the ECR container registry to push images (if applicable):
We can register the image builder and use it in our active stack:
The AWS Image Builder can be customized to a certain extent by providing additional configuration options when registering the image builder. The following additional attributes can be set:
build_image: The Docker image used to build the Docker image. The default is bentolor/docker-dind-awscli, which is a Docker image that includes both Docker-in-Docker and the AWS CLI.
compute_type: The compute type used for the CodeBuild project. The default is BUILD_GENERAL1_SMALL.
custom_env_vars: A dictionary of custom environment variables to be set in the CodeBuild project.
implicit_container_registry_auth: A boolean flag that indicates whether to use implicit or explicit authentication when authenticating the AWS CodeBuild build to the target container registry:
when this is set to true (default), the builds will be configured to use whatever implicit authentication credentials are already available within the build container. As a special case for ECR registries, the service IAM role attached to the CodeBuild project is used to authenticate to the target ECR container registry and therefore the service role must include the necessary permissions to push images to the target ECR registry.
when set to false, the credentials attached to the ZenML Container Registry stack component in the active stack will be set as build environment variables and used to authenticate to the target container registry. This is useful when the target container registry is not an ECR registry or when the service role attached to the CodeBuild project does not have the necessary permissions to push images to the target ECR registry. This works best when the ZenML Container Registry stack component is also linked to the external container registry via a Service Connector.
Recommended: Grant ZenML access to trigger AWS CodeBuild builds by registering an with the proper credentials and permissions, as covered in the section. If not provided, the AWS credentials will be inferred from the environment where the pipeline is triggered.
You also need to set up required to access the CodeBuild AWS service.
Integrating and using an AWS Image Builder in your pipelines is not possible without employing some form of authentication. If you're looking for a quick way to get started locally, you can use the Local Authentication method. However, the recommended way to authenticate to the AWS cloud platform is through . This is particularly useful if you are configuring ZenML stacks that combine the AWS Image Builder with other remote stack components also running in AWS.
This method uses the implicit AWS authentication available in the environment where the ZenML code is running. On your local machine, this is the quickest way to configure an AWS Image Builder. You don't need to supply credentials explicitly when you register the AWS Image Builder, as it leverages the local credentials and configuration that the AWS CLI stores on your local machine. However, you will need to install and set up the AWS CLI on your machine as a prerequisite, as covered in , before you register the AWS Image Builder.
Stacks using the AWS Image Builder set up with local authentication are not portable across environments. To make ZenML pipelines fully portable, it is recommended to use to authenticate your AWS Image Builder to the AWS cloud platform.
To set up the AWS Image Builder to authenticate to AWS and access the AWS CodeBuild services, it is recommended to leverage the many features provided by such as auto-configuration, best security practices regarding long-lived credentials and reusing the same credentials across multiple stack components.
A non-interactive CLI example that leverages on your local machine to auto-configure an AWS Service Connector for the AWS CodeBuild service:
The AWS Service Connector supports with different levels of security and convenience. You should pick the one that best fits your use case.
