Module core.backends.orchestrator.beam.orchestrator_beam_backend¶
Definition of the DataFlow Orchestrator Backend
Classes¶
OrchestratorBeamBackend(worker_machine_type: str = 'e2-medium', num_workers: int = 4, max_num_workers: int = 10, disk_size_gb: int = 100, autoscaling_algorithm: str = 'THROUGHPUT_BASED', **kwargs)
: Uses Apache Beam as a Pipeline orchestrator.
### Ancestors (in MRO)
* zenml.core.backends.orchestrator.local.orchestrator_local_backend.OrchestratorLocalBackend
* zenml.core.backends.base_backend.BaseBackend
### Class variables
`BACKEND_TYPE`
:
### Methods
`run(self, config: Dict[str, Any])`
: This run function essentially calls an underlying TFX orchestrator run.
However it is meant as a higher level abstraction with some
opinionated decisions taken.
Args:
config: a ZenML config dict