Specify pip dependencies and apt packages
The configuration for specifying pip and apt dependencies only works in the remote pipeline case, and is disregarded for local pipelines (i.e. pipelines that run locally without having to build a Docker image).
When a pipeline is run with a remote orchestrator a Dockerfile is dynamically generated at runtime. It is then used to build the Docker image using the image builder component of your stack.
For all of examples on this page, note that DockerSettings
can be imported using from zenml.config import DockerSettings
.
By default, ZenML automatically installs all packages required by your active ZenML stack. However, you can specify additional packages to be installed in various ways:
Install all the packages in your local Python environment (This will use the
pip
orpoetry
package manager to get a list of your local packages):
If required, a custom command can be provided. This command must output a list of requirements following the format of the requirements file:
Specify a list of requirements in code:
Specify a requirements file:
Specify a list of ZenML integrations that you're using in your pipeline:
Specify a list of apt packages in code:
Prevent ZenML from automatically installing the requirements of your stack:
In some cases the steps of your pipeline will have conflicting requirements or some steps of your pipeline will require large dependencies that don't need to be installed to run the remaining steps of your pipeline. For this case, ZenML allows you to specify custom Docker settings for steps in your pipeline.
You can combine these methods but do make sure that your list of requirements does not overlap with the ones specified explicitly in the Docker settings.
Depending on the options specified in your Docker settings, ZenML installs the requirements in the following order (each step optional):
The packages installed in your local python environment
The packages required by the stack unless this is disabled by setting
install_stack_requirements=False
.The packages specified via the
required_integrations
The packages specified via the
requirements
attributeYou can specify additional arguments for the installer used to install your Python packages as follows:
Experimental: If you want to use
uv
for faster resolving and installation of your Python packages, you can use by it as follows:
uv
is a relatively new project and not as stable as pip
yet, which might lead to errors during package installation. If this happens, try switching the installer back to pip
and see if that solves the issue.
Last updated