API Reference

ML Orchestrator package for machine learning pipeline components management.

This package provides tools for creating, parsing, and managing machine learning pipeline components with a focus on Kubeflow Pipelines integration.

class ml_orchestrator.ComponentParser(add_imports: ~typing.List[str] = <factory>, only_function: bool = False)

A parser class that extends FunctionParser to create and manage components.

This class handles components for Kubeflow pipelines (KFP), including decorators and serialized representations.

add_imports

Additional import statements to prepend to output files.

Type:

List[str]

only_function

Whether to generate only the function body.

Type:

bool

classmethod create_decorator(component: MetaComponent) → str

Create a decorator string for a given MetaComponent.

Parameters:

component (MetaComponent) – The component object used to generate a decorator.

Returns:

A decorator string for use in Kubeflow components.

Return type:

str

class ml_orchestrator.EnvironmentParams(base_image: str = None, target_image: str = None, packages_to_install: ~typing.List[str] = <factory>, pip_index_urls: ~typing.List[str] = <factory>, install_kfp_package: bool = None, kfp_package_path: str = None)

A dataclass for defining the configuration parameters used to set up the environment.

This class defines parameters for pipeline components, including Docker images, package installations, and Kubeflow settings.

base_image

The base Docker image to use for building and running pipeline components.

Type:

str

target_image

The Docker image to be built as the finalized environment for pipeline execution.

Type:

str

packages_to_install

A list of additional packages to be installed inside the Docker image.

Type:

List[str]

pip_index_urls

A list of custom pip index URLs that can be used to search for Python packages.

Type:

List[str]

install_kfp_package

Indicates whether the Kubeflow Pipelines (KFP) package should be installed.

Type:

bool

kfp_package_path

The path or URL for the Kubeflow Pipelines (KFP) package to be used for installation.

Type:

str

classmethod comp_fields() → Tuple[Field, ...]

Retrieve all the fields defined in the EnvironmentParams dataclass.

Returns:

A tuple of field definitions.

Return type:

Tuple[dataclasses.Field, …]

comp_vars() → Dict[Field, Any]

Retrieve a mapping of each dataclass field to its corresponding value.

Maps each field to its current value for the current instance.

Returns:

A dictionary mapping fields to their current values.

Return type:

Dict[dataclasses.Field, Any]

class ml_orchestrator.FunctionParser

Handle the parsing and generation of components into KFP (Kubeflow Pipelines) DSL files.

This class is responsible for orchestrating the process of generating string representations of components compatible with KFP DSL, and writing them to files for usage in Kubeflow Pipelines.

create_kfp_file_str(components: List[ComponentProtocol]) → str

Create a KFP DSL string from a list of components.

Parameters:

components – List of components conforming to the ComponentProtocol

Returns:

The generated KFP DSL string for all components

Return type:

str

parse_components_to_file(components: List[ComponentProtocol], filename: str) → None

Parse components into a Python file for use with Kubeflow Pipelines.

Generate a KFP string functions representation of the components and save it to the target file.

Parameters:

• components – List of components conforming to the ComponentProtocol protocol, which defines the structure and behavior required for each component to be processed by this method.

• filename – Name of the file where the generated KFP representation of components will be saved.

class ml_orchestrator.MetaComponent

Concrete MetaComponent with default environment parameters.

This class provides default environment parameters and a method for retrieving the component’s function name in lowercase format.

env

Environment parameters for the component.

Type:

EnvironmentParams

property env: EnvironmentParams

Get the environment parameters for this component.

Returns:

A default EnvironmentParams instance.

Return type:

EnvironmentParams

classmethod kfp_func_name() → str

Get the name of the Kubeflow Pipeline function for this component.

Returns:

The lowercase class name of the component.

Return type:

str