# ADR 0003: Unified declaration model for workflow context bindings

- Status: accepted
- Deciders: Simon
- Date: 2026-05-21

## Context

ADR 0002 settles where the context-stream gate lives (at the JobManager) and the
failure modes it prevents. It does not settle where context streams are *declared*,
nor how a context value reaches the Sciline pipeline. Before this ADR those concerns
were spread across several mechanisms with three sharp edges:

1. **`AuxSources` carried two unrelated concerns.** It was built for user-facing
   dynamic-aux selection (the `AuxInput.choices` widget side). Context streams
   piggybacked on it for routing, declared as plain-string aux entries the UI
   happened to ignore. A routing primitive doubled as a workflow-execution
   primitive.

2. **Two parallel declarations for the same stream.** LOKI's `detector_carriage`
   was declared once for routing and once for graph wiring, with nothing enforcing
   that the names agree.

3. **`Workflow.context_keys` was a protocol leak.** Only `JobFactory.create`
   consulted it, only at job-creation time, yet every `Workflow` implementation had
   to expose it (the actual keys for `StreamProcessorWorkflow`, empty stubs for
   `AreaDetectorView` and `TimeseriesStreamProcessor`). A configuration concern
   leaked into the runtime protocol.

A standing constraint shapes the solution: the dashboard imports workflow `specs.py`
to generate UI, so `specs.py` must stay free of workflow-key imports — the dashboard
must not depend on instrument-specific Sciline pipelines. The hard requirement is the
absence of heavy science-package imports, but the rule is applied as written: even a
dashboard-safe `workflow_key` (a `ValueLog` chain-patch marker, which subclasses a
`config`-layer type) is declared in `factories.py`, co-located with the
`add_context_binding` that consumes it, rather than in `specs.py`.

### Param-dependent context — explicit non-goal

Detector-view and monitor-view factories wire position context unconditionally for
sources that have a motion binding, regardless of `coordinate_mode` (TOA vs
wavelength). A precise design would gate only in wavelength mode (TOA does not consume
position). We accept the over-gate: an instrument wiring a stream as context must keep
the producer healthy. If "plot raw detector with a broken motion controller" becomes a
real operational need, the answer is to split the spec into a TOA-only and a wavelength
variant, not to introduce param-dependent gating. Today this is hypothetical; YAGNI.

## Decision

One declaration record, `ContextBinding`, declarable at two scopes, feeds both the
factory's `context_keys` (wired into `set_context`) and the job's gating set (ADR
0002).

```python
@dataclass(frozen=True, slots=True, kw_only=True)
class ContextBinding:
    stream_name: str                 # internal stream name; the data-path lookup key into context_keys
    workflow_key: Any                # opaque Sciline graph key, consumed by the factory
    dependent_sources: frozenset[str]
```

It is a single concrete dataclass — no subclasses, no resolver/seed callables. Two
flavours share the record, discriminated by the *type* of `workflow_key`:

- **Direct parameter.** `workflow_key` is an arbitrary Sciline key (e.g.
  `InstrumentAngle[SampleRun]`). The stream value is fed straight into
  `set_context`.
- **NXtransformations chain patch.** `workflow_key` is a `ValueLog` subclass. The
  value is patched into the NeXus `depends_on` chain rather than bound directly.
  `wire_dynamic_transforms` discovers these by
  `issubclass(workflow_key, ValueLog)`. The chain entry to patch is *derived* from
  `stream_name` (the f144 substream's `nexus_path`) via `Instrument.chain_patch_path`
  — there is no separate `transform_path` field, keeping a single source of truth in
  the parsed stream definitions.

### Two declaration sites

1. **`Instrument.add_context_binding(*, stream_name, dependent_sources, workflow_key)`**
   — source-scoped. Stored on `Instrument.context_bindings`. Used for instrument-property
   context (motion/geometry). Chain-patch bindings live at instrument scope only.

2. **`SpecHandle.add_context_binding(stream_name, workflow_key, dependent_sources=None)`**
   — spec-scoped. Late-bound from `factories.py` (mirroring `attach_factory()`),
   keeping `specs.py` free of workflow-key imports. `dependent_sources` defaults to the
   spec's `source_names`. Stored on `WorkflowRegistration.context_bindings`. Rejects
   `ValueLog` keys: chain-patch is instrument-scope. There are no callables — the wire
   name equals `stream_name` and there is no seed, so the gate stays closed until the
   producer publishes (correct for a context with no safe default).

### Resolution at job creation

`Instrument.resolve_context_keys(workflow_id, source_name)` merges instrument- and
spec-scope bindings filtered by source membership and returns
`{stream_name: workflow_key}`:

```python
instrument_bindings = [] if registration.skip_instrument_contexts \
    else self.context_bindings
return {binding.stream_name: binding.workflow_key
        for binding in (*instrument_bindings, *registration.context_bindings)
        if source_name in binding.dependent_sources}
```

`JobFactory.create` calls it once per job. The resolved `{stream_name: workflow_key}`
mapping is the single source of the gate ADR 0002 applies: it goes to the factory as
`context_keys`, and its key set becomes `Job.gating_streams` — the streams whose values
must be present before the job runs (context wire names equal their stream names). This
is where "which streams are gated" is decided: the gated set *is* the set of resolved
context bindings for the `(spec, source)`, nothing more. The merge lives on `Instrument`
rather than inlined in `JobFactory` so workflow visualization can resolve the same keys
without constructing a job.
`JobFactory.create` returns a bare `Job`. `AuxSources` is slimmed to dynamic,
user-selectable aux only — no context-flavoured entries, no `initial_context_messages`.

### `skip_instrument_contexts` opt-out

Motion is a property of the *source*, but not every spec on that source consumes the
geometry value (LOKI `tube_view`, bifrost `detector_ratemeter`, the bifrost detector
view). Such a spec declares `SpecHandle.skip_instrument_contexts()`, which drops
instrument-scope bindings for that spec's jobs (spec-scope bindings are unaffected).
The opt-out is called from `factories.py`, co-located with the
`Instrument.add_context_binding` it negates: the flag is meaningless without the
binding it cancels, and the rationale is implementation knowledge. It needs no
workflow-key import, so — unlike `add_context_binding` — its placement is not forced
by the `specs.py` import constraint; co-location is a legibility choice.

Instrument scope as the default — with an opt-out — is chosen over spec scope as the
default deliberately. Spec-scope-by-default makes "added a new spec on the source,
forgot to declare carriage" a *silently-wrong* failure (the workflow runs with stale
geometry). Instrument-scope-plus-opt-out makes the symmetric mistake *noisy*: a
forgotten `skip_instrument_contexts()` gates a workflow on a stream it never reads, so
the workflow visibly never runs rather than producing wrong output.

### Chain patching

`ValueLog` lives in `config/value_log.py`, next to `ContextBinding`, so `config` does
not import `handlers`. Each chain-patch binding declares its own `ValueLog` subclass,
giving it a distinct Sciline parameter so multiple dynamic transforms can coexist on
one workflow without colliding.

`Instrument.chain_patch_bindings` selects the instrument-scope chain-patch
`ContextBinding` records (those whose `workflow_key` is a `ValueLog` subclass),
resolves each one's NeXus transform path via `Instrument.chain_patch_path`, and returns
them as `ChainPatchBinding` records — a *resolved form* of the declaration, not a second
declaration type, carrying `transform_path` alongside the `ValueLog` key and
`dependent_sources`. `wire_dynamic_transforms(workflow, bindings)` is then a pure
function over those records: it reads each input's component type off the workflow's own
`dynamic_keys`, groups the matching bindings by component type, and builds one fused
per-component provider (`build_patched_chain_provider` / `synthesise_provider` in
`handlers/dynamic_transforms.py`) that replaces essreduce's
`NeXusTransformationChain[T, SampleRun]` provider and writes the latest sample of each
`ValueLog` parameter into the chain. Resolving the path up front keeps the wiring step
free of any reference to the instrument's stream topology. The same seam works for the
detector-view factory and for a pre-built `LokiWorkflow()`.

### Routing pickup

Routing (the Kafka subscription set) is decided per *namespace* at service startup,
ahead of any specific job, and may over-subscribe; the bandwidth cost of an unused
f144 stream is negligible. Gating is decided per *job* at `JobFactory.create` and must
be precise. `config/route_derivation.py:gather_source_names` therefore includes context
bindings in the subscription set: spec-scope entries by stream name, instrument-scope
entries when a spec in the namespace shares a source with the binding's
`dependent_sources`. This pickup is a co-requirement of this ADR, not a follow-up:
without it the namespace never subscribes to motion streams and the gate stays closed
indefinitely.

### Workflow protocol stays pure

`Workflow.context_keys` is removed from the protocol. The routing layer injects the
resolved `context_keys` via `SupportsContext.build` *after* the factory returns the
workflow, so factories never declare `context_keys` in their signature;
`StreamProcessorWorkflow` defers constructing the wrapped `StreamProcessor` until
`build` so the keys can be merged before the graph is baked (the uniform-keying and
deferred-build seam is settled in
ADR 0004). `AreaDetectorView` and `TimeseriesStreamProcessor` lose their empty-stub
property. The `Workflow` protocol reduces to `accumulate`, `finalize`, `clear`;
context handling is a `StreamProcessorWorkflow`-internal concern, opaque to `Job`
and `JobManager`.

## Alternatives considered

| Option | Notes |
|---|---|
| **One `ContextBinding` record, two declaration sites (chosen)** | Single type covers both scopes and both flavours. Late-binding for the spec side keeps `specs.py` workflow-key-free. |
| Separate types per scope or per flavour (subclass split) | Overlapping shapes whose differences are in *defaults*, not data; a hierarchy adds friction without paying its way. Rejected. |
| One declaration on `WorkflowSpec` as a constructor field carrying `workflow_key` | Drags workflow-key imports into `specs.py` and therefore the dashboard. Rejected. |
| Spec lists stream fields; factory declares keys separately; cross-reference by name | Two declarations must still agree on names; restores the routing-vs-factory drift in a new form. Rejected. |
| Spec scope as default for motion | Makes a forgotten declaration silently-wrong (stale geometry). Instrument-scope-plus-opt-out makes it noisy instead. Rejected. |
| Per-source `transform_name` dict local to `loki/factories.py` | The chain path is derived from `stream_name` via `chain_patch_path`; a parallel dict would re-introduce manual sync. Rejected. |
| Keep `Workflow.context_keys` as the gate source | Leaves the `AuxSources`/factory duplication and the protocol leak in place. Rejected. |

## Key design choices

### Workflow keys stay out of `specs.py` via late-binding

`SpecHandle.add_context_binding(...)` is invoked from `factories.py`, the module that
owns `attach_factory()`. Dashboards importing `specs.py` to render UI never see a
workflow-key import.

### Two scopes, one record shape

Instrument scope is for source properties (motion), where chain-patch also lives. Spec
scope is for context that is a property of one workflow. The spec-scope path has no
current user — it is the extension point for the motivating future case, a
**sample-temperature** stream feeding one reduction workflow: gated (no safe default),
a real shared Kafka stream (wire name == `stream_name`), no seed. That drops straight
into `SpecHandle.add_context_binding` with no further machinery.

### Chain path derived, not declared

The NeXus transformation-chain entry a chain-patch binding writes is derived from its
`stream_name` via `Instrument.chain_patch_path`, so the f144 substream definition is the
single source of truth and a binding cannot disagree with the stream it patches.

## Consequences

- `ContextBinding` lives in `config/stream.py` as a single concrete dataclass
  (`stream_name`, `workflow_key`, `dependent_sources`). Its resolved chain-patch
  form `ChainPatchBinding` (adding `transform_path`) lives alongside it; `ValueLog`
  lives in `config/value_log.py`.
- `Instrument` carries `context_bindings: list[ContextBinding]` and
  `add_context_binding(...)`, and exposes `chain_patch_bindings` (the resolved
  `ChainPatchBinding` records); `wire_dynamic_transforms(workflow, bindings)` patches
  those into the pipeline.
- `WorkflowRegistration` carries `context_bindings` and a `skip_instrument_contexts`
  flag; `SpecHandle` gains `add_context_binding(...)` and `skip_instrument_contexts()`.
- `Instrument.resolve_context_keys` merges instrument + spec bindings; `JobFactory.create`
  calls it, hands `context_keys` to the factory (injected via `SupportsContext.build`,
  ADR 0004), sets `Job.gating_streams`, and returns a bare `Job`.
- `AuxSources` is dynamic, user-selectable aux only. ROI remains an `AuxSources`
  (`DetectorROIAuxSources`, job-prefixed per job) — it is not a context binding, since
  ADR 0002 routes ROI as aux rather than gating it.
- `Workflow.context_keys` is removed from the protocol; `StreamProcessorWorkflow`
  receives the routing
  layer's `context_keys` via `build` (ADR 0004), other implementations drop the stub.
- `gather_source_names` includes context bindings so the namespace subscribes to gated
  streams. Without it the gate never opens for motion.
- Validators enforce the record's invariants: chain-patch `(stream_name, workflow_key)`
  uniqueness; wire-name collisions across instrument/spec scope and against aux field
  names; `dependent_sources` against registered spec `source_names`. `synthesise_provider`
  validates identifier names before splicing into its `exec` template.
- `tests/config/motion_binding_test.py` walks every chain-patch binding's derived
  transform path against the registered NeXus geometry artifact and fails if the path is
  not on the declared consumer's `depends_on` chain — closing wrong-path as a silent
  failure without putting NeXus introspection on the construction hot path.
- Per ADR 0002, the gate mechanism inside `JobManager` is unchanged; this ADR settles
  only the declaration model that feeds it.