ZenMLBeamlDagRunner(beam_orchestrator_args: Union[List[str], NoneType] = None, config: Union[tfx.orchestration.config.pipeline_config.PipelineConfig, NoneType] = None)
: This is the same as the super class from tfx:
tfx.orchestration.beam.beam_dag_runner.BeamDagRunner with the exception
being that the pipeline_run is not overridden. Full credit to Google LLC
for the original source code found at:
Initializes BeamDagRunner as a TFX orchestrator. Args: beam_orchestrator_args: beam args for the beam orchestrator. Note that this is different from the beam_pipeline_args within additional_pipeline_args, which is for beam pipelines in components. config: Optional pipeline config for customizing the launching of each component. Defaults to pipeline config that supports InProcessComponentLauncher and DockerComponentLauncher. ### Ancestors (in MRO) * tfx.orchestration.beam.beam_dag_runner.BeamDagRunner * tfx.orchestration.tfx_runner.TfxRunner ### Methods `run(self, tfx_pipeline: tfx.orchestration.pipeline.Pipeline) ‑> NoneType` : Deploys given logical pipeline on Beam. Args: tfx_pipeline: Logical pipeline containing pipeline args and components.