Backblaze B2

Storing artifacts in a Backblaze B2 cloud storage bucket.

The Backblaze B2 Artifact Store is an Artifact Store flavor provided with the B2 ZenML integration that uses Backblaze B2 Cloud Storage to store ZenML artifacts. Backblaze B2 exposes an S3-compatible API, so this flavor reuses ZenML's S3 implementation under the hood while adding B2-friendly runtime fallbacks for the endpoint URL and credential environment variables.

When would you want to use it?

Running ZenML pipelines with the local Artifact Store is usually sufficient if you just want to evaluate ZenML or get started quickly without incurring the trouble and the cost of employing cloud storage services in your stack. However, the local Artifact Store becomes insufficient or unsuitable if you have more elaborate needs for your project:

• if you want to share your pipeline run results with other team members or stakeholders inside or outside your organization
• if you have other components in your stack that are running remotely (e.g. a Kubeflow or Kubernetes Orchestrator running in a public cloud).
• if you outgrow what your local machine can offer in terms of storage space and need to use some form of private or public storage service that is shared with others
• if you are running pipelines at scale and need an Artifact Store that can handle the demands of production-grade MLOps

You should use the Backblaze B2 Artifact Store when you want to keep your ZenML artifacts in shared object storage and you prefer Backblaze B2's pricing, egress terms, or geographic regions over the hyperscalers. Because the integration speaks B2's S3-compatible API, you get the same workflow as the S3 Artifact Store without having to manage any additional B2-native tooling. You should consider one of the other Artifact Store flavors if you don't have a Backblaze B2 account or if you require an S3 service that is co-located with other AWS resources in your stack.

How do you deploy it?

The Backblaze B2 Artifact Store flavor is provided by the B2 ZenML integration, you need to install it on your local machine to be able to register a B2 Artifact Store and add it to your stack:

zenml integration install b2 -y

You will also need a Backblaze B2 account and a private bucket to hold your ZenML artifacts. The high-level setup steps in the Backblaze console are:

1. Create a B2 bucket. From the Backblaze web console, navigate to B2 Cloud Storage > Buckets and click Create a Bucket. Pick a globally-unique bucket name (this becomes the s3://<bucket-name> path used by ZenML), choose Private, and select the region you want to store data in. The region segment of the bucket's S3 endpoint (e.g. us-west-004, eu-central-003) is needed in the next step.
2. Generate a B2 application key. Navigate to Account > Application Keys and click Add a New Application Key. Scope the key to the bucket you just created and grant at minimum Read and Write access. Backblaze will display a keyID and an applicationKey exactly once — copy both values immediately.
3. Export the credentials as environment variables. ZenML's B2 artifact store reads Backblaze's standard environment variable names at runtime if no key/secret is configured directly on the artifact store:

   export B2_APPLICATION_KEY_ID=<your-b2-key-id>
   export B2_APPLICATION_KEY=<your-b2-application-key>

   This convention matches the Backblaze CLI and SDKs, so the same environment that authenticates b2-cli will authenticate ZenML.

For full Backblaze documentation on bucket creation and application keys, see the B2 Cloud Storage docs.

How do you use it?

The only configuration parameter mandatory for registering a B2 Artifact Store is the root path URI, which needs to point to a B2 bucket using the s3:// scheme — this is intentional, because the B2 flavor talks to Backblaze through its S3-compatible API:

# Register the B2 artifact store (uses the s3:// URI scheme since B2 speaks the S3 API)
zenml artifact-store register my_b2 --flavor=b2 --path=s3://my-bucket/zenml-artifacts

# Register and set a stack that uses the new artifact store
zenml stack register custom_stack -a my_b2 ... --set

If B2_APPLICATION_KEY_ID and B2_APPLICATION_KEY are exported in the shell that runs ZenML, no further authentication configuration is required. The artifact store client will pick them up when it is created without saving them as component configuration. To keep credentials out of the shell environment, store them in a ZenML Secret and reference them on registration:

# Store the B2 application key in a ZenML secret
zenml secret create b2_secret \
    --access_key_id='<YOUR_B2_APPLICATION_KEY_ID>' \
    --secret_access_key='<YOUR_B2_APPLICATION_KEY>'

# Register the B2 artifact store and reference the secret
zenml artifact-store register my_b2 --flavor=b2 \
    --path=s3://my-bucket/zenml-artifacts \
    --authentication_secret=b2_secret

Endpoint defaults and regions

Configure the endpoint that matches your B2 bucket region at registration time:

zenml artifact-store register my_b2 --flavor=b2 \
    --path=s3://my-bucket/zenml-artifacts \
    --client_kwargs='{"endpoint_url": "https://s3.eu-central-003.backblazeb2.com"}'

Use the S3 endpoint shown for your bucket in the Backblaze console. It always has the form https://s3.<region>.backblazeb2.com. If you omit client_kwargs.endpoint_url, the runtime client falls back to https://s3.us-west-004.backblazeb2.com without saving that fallback on the component configuration.

Advanced configuration

Because the B2 flavor subclasses the S3 flavor, every advanced option supported by the S3 Artifact Store — client_kwargs, config_kwargs, and s3_additional_kwargs — is available here too and is passed transparently to the underlying s3fs / botocore stack. Use these to configure things like server-side encryption, custom retry behavior, or alternative B2 endpoints.

Aside from the fact that the artifacts are stored in Backblaze B2, using the B2 Artifact Store is no different from using any other flavor of Artifact Store.