Trigger a pipeline from REST API

Trigger a pipeline from the rest API.

This is a ZenML Pro only feature. Please sign up here get access. OSS users can only trigger a pipeline by calling the pipeline function inside their runner script.

Triggering a pipeline from the REST API only works with pipelines that are already built with a remote stack (i.e. at least a remote orchestrator, artifact store, and container registry). Most of the times, this means you can only re-run a pipeline with a remote stack using a different configuration.

As a pre-requisite, you need a pipeline name and version. After you have it, there are three calls that need to be made in order to trigger a pipeline from the REST API:

1. GET /pipelines?name=<PIPELINE_NAME>&version=<PIPELINE_VERSION> -> This returns a response, where a <PIPELINE_ID> can be copied

2. GET /pipeline_builds?pipeline_id=<PIPELINE_ID> -> This returns a list of responses where a <BUILD_ID> can be chosen

3. POST /pipeline_builds/<BUILD_ID>/runs -> This runs the pipeline. You can pass the PipelineRunConfiguration in the body

The above set of REST calls means that you can only trigger a pipeline that has been built before on a remote stack.

A worked example

Learn how to get a bearer token for the curl commands here.

Here is an example. Let's say would we like to re-run a pipeline called training. We first query the /pipelines endpoint:

curl -X 'GET' \
  '<YOUR_ZENML_SERVER_URL>/api/v1/pipelines?hydrate=false&name=training&version=45' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <YOUR_TOKEN>'

Identifying the pipeline ID

We can take the ID from any object in the list of responses. In this case, the <PIPELINE_ID> is c953985e-650a-4cbf-a03a-e49463f58473 in the response.

After this, we take the pipeline ID and call the /pipeline_builds?pipeline_id=<PIPELINE_ID> API:

curl -X 'GET' \
  '<YOUR_ZENML_SERVER_URL>/api/v1/pipeline_builds?hydrate=false&logical_operator=and&page=1&size=20&pipeline_id=b826b714-a9b3-461c-9a6e-1bde3df3241d' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <YOUR_TOKEN>'

We can now take the <BUILD_ID> from this response. Here it is b826b714-a9b3-461c-9a6e-1bde3df3241d.

Identifying the build ID

Finally, we can use the build ID to trigger the pipeline with a different configuration:

curl -X 'POST' \
  '<YOUR_ZENML_SERVER_URL>/api/v1/pipeline_builds/b826b714-a9b3-461c-9a6e-1bde3df3241d/runs' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer <YOUR_TOKEN>' \
  -d '{
  "steps": {"model_trainer": {"parameters": {"model_type": "rf"}}}
}'

A positive response means your pipeline has been re-triggered with a different config!

Learn how to run a pipeline directly from another

Learn how to run a pipeline directly from the Python client