Local Artifact Store

Storing artifacts on your local filesystem.

The local Artifact Store is a built-in ZenML Artifact Store flavor that uses a folder on your local filesystem to store artifacts.

When would you want to use it?

The local Artifact Store is a great way to get started with ZenML, as it doesn't require you to provision additional local resources or to interact with managed object-store services like Amazon S3 and Google Cloud Storage. All you need is the local filesystem. You should use the local Artifact Store if you're just evaluating or getting started with ZenML, or if you are still in the experimental phase and don't need to share your pipeline artifacts (dataset, models, etc.) with others.

The local Artifact Store is not meant to be utilized in production. The local filesystem cannot be shared across your team and the artifacts stored in it cannot be accessed from other machines. This also means that artifact visualizations will not be available when using a local Artifact Store through a ZenML instance deployed in the cloud.

Furthermore, the local Artifact Store doesn't cover services like high-availability, scalability, backup and restore and other features that are expected from a production grade MLOps system.

The fact that it stores artifacts on your local filesystem also means that not all stack components can be used in the same stack as a local Artifact Store:

• only Orchestrators running on the local machine, such as the local Orchestrator, a local Kubeflow Orchestrator, or a local Kubernetes Orchestrator can be combined with a local Artifact Store

• only Model Deployers that are running locally, such as the MLflow Model Deployer, can be used in combination with a local Artifact Store

• Step Operators: none of the Step Operators can be used in the same stack as a local Artifact Store, given that their very purpose is to run ZenML steps in remote specialized environments

As you transition to a team setting or a production setting, you can replace the local Artifact Store in your stack with one of the other flavors that are better suited for these purposes, with no changes required in your code.

How do you deploy it?

The default stack that comes pre-configured with ZenML already contains a local Artifact Store:

$ zenml stack list
Running without an active repository root.
Using the default local database.
┏━━━━━━━━┯━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━┓
┃ ACTIVE │ STACK NAME │ ARTIFACT_STORE │ ORCHESTRATOR ┃
┠────────┼────────────┼────────────────┼──────────────┨
┃   👉   │ default    │ default        │ default      ┃
┗━━━━━━━━┷━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━┛

$ zenml artifact-store describe
Running without an active repository root.
Using the default local database.
Running with active stack: 'default'
No component name given; using `default` from active stack.
                           ARTIFACT_STORE Component Configuration (ACTIVE)                           
┏━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ COMPONENT_PROPERTY │ VALUE                                                                        ┃
┠────────────────────┼──────────────────────────────────────────────────────────────────────────────┨
┃ TYPE               │ artifact_store                                                               ┃
┠────────────────────┼──────────────────────────────────────────────────────────────────────────────┨
┃ FLAVOR             │ local                                                                        ┃
┠────────────────────┼──────────────────────────────────────────────────────────────────────────────┨
┃ NAME               │ default                                                                      ┃
┠────────────────────┼──────────────────────────────────────────────────────────────────────────────┨
┃ UUID               │ 2b7773eb-d371-4f24-96f1-fad15e74fd6e                                         ┃
┠────────────────────┼──────────────────────────────────────────────────────────────────────────────┨
┃ PATH               │ /home/stefan/.config/zenml/local_stores/2b7773eb-d371-4f24-96f1-fad15e74fd6e ┃
┗━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛

As shown by the PATH value in the zenml artifact-store describe output, the artifacts are stored inside a folder on your local filesystem.

You can create additional instances of local Artifact Stores and use them in your stacks as you see fit, e.g.:

# Register the local artifact store
zenml artifact-store register custom_local --flavor local

# Register and set a stack with the new artifact store
zenml stack register custom_stack -o default -a custom_local --set

Same as all other Artifact Store flavors, the local Artifact Store does take in a path configuration parameter that can be set during registration to point to a custom path on your machine. However, it is highly recommended that you rely on the default path value, otherwise, it may lead to unexpected results. Other local stack components depend on the convention used for the default path to be able to access the local Artifact Store.

For more, up-to-date information on the local Artifact Store implementation and its configuration, you can have a look at the SDK docs .

How do you use it?

Aside from the fact that the artifacts are stored locally, using the local Artifact Store is no different from using any other flavor of Artifact Store.