Develop a custom data validator

How to develop a custom data validator

Before diving into the specifics of this component type, it is beneficial to familiarize yourself with our general guide to writing custom component flavors in ZenML. This guide provides an essential understanding of ZenML's component flavor concepts.

ZenML comes equipped with Data Validator implementations that integrate a variety of data logging and validation libraries, frameworks and platforms. However, if you need to use a different library or service as a backend for your ZenML Data Validator, you can extend ZenML to provide your own custom Data Validator implementation.

Build your own custom data validator

If you want to implement your own custom Data Validator, you can follow the following steps:

  1. Create a class which inherits from the BaseDataValidator class and override one or more of the abstract methods, depending on the capabilities of the underlying library/service that you want to integrate.

  2. If you need any configuration, you can create a class which inherits from the BaseDataValidatorConfig class.

  3. Bring both of these classes together by inheriting from the BaseDataValidatorFlavor.

  4. (Optional) You should also provide some standard steps that others can easily insert into their pipelines for instant access to data validation features.

Once you are done with the implementation, you can register it through the CLI. Please ensure you point to the flavor class via dot notation:

zenml data-validator flavor register <path.to.MyDataValidatorFlavor>

For example, if your flavor class MyDataValidatorFlavor is defined in flavors/my_flavor.py, you'd register it by doing:

zenml data-validator flavor register flavors.my_flavor.MyDataValidatorFlavor

Afterwards, you should see the new flavor in the list of available flavors:

zenml data-validator flavor list
ZenML Scarf

Last updated

Was this helpful?