Module core.backends.orchestrator.beam.zenml_beam_orchestrator¶
Classes¶
ZenMLBeamlDagRunner(beam_orchestrator_args: Union[List[str], NoneType] = None, config: Union[tfx.orchestration.config.pipeline_config.PipelineConfig, NoneType] = None)
: This is the same as the super class from tfx:
tfx.orchestration.beam.beam_dag_runner.BeamDagRunner with the exception
being that the pipeline_run is not overridden. Full credit to Google LLC
for the original source code found at:
https://github.com/tensorflow/tfx/tree/master/tfx/orchestration/beam/
Initializes BeamDagRunner as a TFX orchestrator.
Args:
beam_orchestrator_args: beam args for the beam orchestrator. Note that
this is different from the beam_pipeline_args within
additional_pipeline_args, which is for beam pipelines in components.
config: Optional pipeline config for customizing the launching of each
component. Defaults to pipeline config that supports
InProcessComponentLauncher and DockerComponentLauncher.
### Ancestors (in MRO)
* tfx.orchestration.beam.beam_dag_runner.BeamDagRunner
* tfx.orchestration.tfx_runner.TfxRunner
### Methods
`run(self, tfx_pipeline: tfx.orchestration.pipeline.Pipeline) ‑> NoneType`
: Deploys given logical pipeline on Beam.
Args:
tfx_pipeline: Logical pipeline containing pipeline args and components.