Connect your git repository
Tracking your code and avoiding unnecessary docker builds by connecting your git repo.
Connect your git repository
Speeding up Docker builds for containerized components
It is also important to take some additional points into consideration:
The file download is only possible if the local checkout is clean (i.e. it does not contain any untracked or uncommitted files) and the latest commit has been pushed to the remote repository. This is necessary as otherwise, the file download inside the Docker container will fail.
In order to benefit from the advantages of having a code repository in a project, you need to make sure that the relevant integrations are installed for your ZenML installation.
For instance, let's assume you are working on a project with ZenML and one of your team members has already registered a corresponding code repository of type github
for it. If you do zenml code-repository list
, you would also be able to see this repository. However, in order to fully use this repository, you still need to install the corresponding integration for it, in this example the github
integration.
Registering a code repository
Afterward, code repositories can be registered using the CLI:
Detecting local code repository checkouts
Once you have registered one or more code repositories, ZenML will check whether the files you use when running a pipeline are tracked inside one of those code repositories. This happens as follows:
Next, ZenML checks whether this source root directory is included in a local checkout of one of the registered code repositories
Tracking code version for pipeline runs
Available implementations
GitHub
ZenML provides built-in support for using GitHub as a code repository for your ZenML pipelines. You can register a GitHub code repository by providing the URL of the GitHub instance, the owner of the repository, the name of the repository, and a GitHub Personal Access Token (PAT) with access to the repository.
Before registering the code repository, first, you have to install the corresponding integration:
Afterward, you can register a GitHub code repository by running the following CLI command:
where is the name of the code repository you are registering, is the owner of the repository, is the name of the repository, <GITHUB_TOKEN> is your GitHub Personal Access Token and <GITHUB_URL> is the URL of the GitHub instance which defaults to https://github.com.
You will need to set a URL if you are using GitHub Enterprise.
After registering the GitHub code repository, ZenML will automatically detect if your source files are being tracked by GitHub and store the commit hash for each pipeline run.
GitLab
ZenML also provides built-in support for using GitLab as a code repository for your ZenML pipelines. You can register a GitLab code repository by providing the URL of the GitLab project, the group of the project, the name of the project, and a GitLab Personal Access Token (PAT) with access to the project.
Before registering the code repository, first, you have to install the corresponding integration:
Afterward, you can register a GitLab code repository by running the following CLI command:
where is the name of the code repository you are registering, is the group of the project, is the name of the project, <GITLAB_TOKEN> is your GitLab Personal Access Token, and <GITLAB_URL> is the URL of the GitLab instance which defaults to https://gitlab.com.
You will need to set a URL if you have a self-hosted GitLab instance.
After registering the GitLab code repository, ZenML will automatically detect if your source files are being tracked by GitLab and store the commit hash for each pipeline run.
Developing a custom code repository
If you're using some other platform to store your code and you still want to use a code repository in ZenML, you can implement and register a custom code repository.
First, you'll need to subclass and implement the abstract methods of the zenml.code_repositories.BaseCodeRepository
class:
After you're finished implementing this, you can register it as follows:
Last updated