sciline.Pipeline

class sciline.Pipeline(providers=None, *, params=None)

A container for providers that can be assembled into a task graph.

__init__(providers=None, *, params=None)

Setup a Pipeline from a list providers

Parameters:

• providers (Optional[Iterable[Callable[..., Any] | Provider]], default: None) – List of callable providers. Each provides its return value. Their arguments and return value must be annotated with type hints.

• params (Optional[dict[type[Any], Any]], default: None) – Dictionary of concrete values to provide for types.

Methods

__init__([providers, params])	Setup a Pipeline from a list providers
bind_and_call(fns, /)	Call the given functions with arguments provided by the pipeline.
compute(tp, **kwargs)	Compute result for the given keys.
copy()	rtype: TypeVar(T, bound= DataGraph)
get(keys, *[, scheduler, handler])	Return a TaskGraph for the given keys.
insert(provider, /)	Insert a callable into the graph that provides its return value.
map(node_values)	rtype: TypeVar(T, bound= DataGraph)
output_keys()	Returns the keys that are not inputs to any other providers.
reduce(*, func, **kwargs)	rtype: TypeVar(T, bound= DataGraph)
to_networkx()	rtype: DiGraph
visualize([tp, compact, mode, ...])	Return a graphviz Digraph object representing the graph for the given keys.
visualize_data_graph(**kwargs)	rtype: graphviz.Digraph

Attributes

index_names	Names of the indices (dimensions) of the graph.
indices	Names and values of the indices of the graph.
underlying_graph	The underlying NetworkX graph.

bind_and_call(fns, /)

Call the given functions with arguments provided by the pipeline.

Parameters:

fns (Callable[..., Any] | Iterable[Callable[..., Any]]) –

Functions to call. The pipeline will provide all arguments based on the function’s type hints.

If this is a single callable, it is called directly. Otherwise, bind_and_call will iterate over it and call all functions. If will in either case call Pipeline.compute() only once.

Returns:

Any – The return values of the functions in the same order as the functions. If only one function is passed, its return value is not wrapped in a tuple.

compute(tp, **kwargs)

Compute result for the given keys.

Equivalent to self.get(tp).compute().

Parameters:

• tp (:sphinx_autodoc_typehints_type:`:py:class:`type` | :py:class:`~collections.abc.Iterable`\ \[:py:class:`type`] | `) – Type to compute the result for. Can be a single type or an iterable of types.

• kwargs (Any) – Keyword arguments passed to the .get() method.

Return type:

Any

get(keys, *, scheduler=None, handler=None)

Return a TaskGraph for the given keys.

Parameters:

• keys (:sphinx_autodoc_typehints_type:`:py:class:`type` | :py:class:`~collections.abc.Iterable`\ \[:py:class:`type`] | `) – Type to compute the result for. Can be a single type or an iterable of types.

• scheduler (Optional[Scheduler], default: None) – Optional scheduler to use for computing the result. If not given, a NaiveScheduler is used if dask is not installed, otherwise dask’s threaded scheduler is used.

• handler (Optional[ErrorHandler], default: None) – Handler for unsatisfied requirements. If not provided, HandleAsBuildTimeException is used, which raises an exception. During development and debugging it can be helpful to use a handler that raises an exception only when the graph is computed. This can be achieved by passing HandleAsComputeTimeException as the handler.

Return type:

TaskGraph

output_keys()

Returns the keys that are not inputs to any other providers.

Return type:

tuple[type, ...]

visualize(tp=None, compact=False, mode='data', cluster_generics=True, cluster_color='#f0f0ff', **kwargs)

Return a graphviz Digraph object representing the graph for the given keys.

Equivalent to self.get(tp).visualize().

Parameters:

• tp (Union[type, Iterable[type], None], default: None) – Type to visualize the graph for. Can be a single type or an iterable of types.

• compact (bool, default: False) – If True, parameter-table-dependent branches are collapsed into a single copy of the branch. Recommended for large graphs with long parameter tables.

• mode (Literal['data', 'task', 'both'], default: 'data') – If ‘data’, only data nodes are shown. If ‘task’, only task nodes and input data nodes are shown. If ‘both’, all nodes are shown.

• cluster_generics (bool, default: True) – If True, generic products are grouped into clusters.

• cluster_color (str | None, default: '#f0f0ff') – Background color of clusters. If None, clusters are dotted.

• kwargs (Any) – Keyword arguments passed to graphviz.Digraph.

Return type:

Digraph