# ADR 0009: Generate the per-instrument stream registry at development time

- Status: accepted
- Deciders: Simon
- Date: 2026-07-22

This ADR is retroactive: it records a decision already implemented, written up after the fact.

## Context

NeXus-rich instruments (bifrost, dream, estia, loki, nmx, odin, tbl) each carry O(100) f144
log streams — motion, sample environment, choppers. Every one needs its NeXus path, Kafka
`topic`, `source`, and `units` recorded as an `F144Stream` in `Instrument.streams`
(`src/ess/livedata/config/stream.py`) before the corresponding workflow can be wired up.
That data already exists, authoritatively, as group attributes in the instrument's NeXus
geometry file — the same file the filewriter and `essreduce` consume.

Both the backend services and the dashboard import an instrument's `config.instruments.<inst>`
package at startup to build `Instrument`. The dashboard in particular has no file-access path
at that point: it runs wherever a user's browser can reach it, not necessarily co-located with
a NeXus geometry file or a Pooch cache.

## Decision

Extract the f144 stream declarations from a NeXus geometry file *offline*, via the CLI in
`nexus_helpers.py` (`python -m ess.livedata.nexus_helpers <geometry.nxs> --generate`), and
check the generated `config/instruments/<inst>/streams_parsed.py` into git as an ordinary
Python module. The instrument's hand-edited `specs.py` imports the generated
`PARSED_STREAMS` dict and composes the final registry via `name_streams`, supplying renames
and any hand-written synthetic entries.

Instruments with few streams (today: `dummy`) skip codegen entirely and hand-write the
`dict[str, F144Stream]` literal.

## Alternatives considered

| Option | Notes |
|---|---|
| **Codegen from NeXus at development time, checked in (chosen)** | Both backend and dashboard start from a plain importable module — no file I/O, no network, no cache-state dependency at import time. The generated diff is reviewable like any other code change against the previous geometry. |
| Parse the NeXus geometry file at process startup | The dashboard has no guaranteed file-access path to the geometry file; it would need a `Pooch` download, adding cold-start latency and a runtime dependency on cache state and network reachability that the dashboard does not otherwise have. Rejected. |
| Hand-maintain the registry (YAML or Python) for every instrument | O(100) entries per instrument, sourced from filewriter attributes nobody wants to retype. Selectively already done for `dummy`, where the entry count is small enough that hand-maintenance is cheaper than adding a geometry file just to run codegen. Rejected as the general approach. |

## Key design choices

### Generated output is a plain Python module, not a data file

`generate_streams_parsed_module` emits a `.py` file with an `F144Stream` dict literal rather
than YAML or JSON. It imports and type-checks like any other module, needs no bespoke parser
in `specs.py`, and diffs the same way as hand-written code in review.

### The generated dict is keyed by NeXus path, not instrument-facing name

`streams_parsed.py` only extracts what the geometry file states directly; assigning
instrument-facing names (auto-suggestion, collision handling, device detection, renames) is
`name_streams`'s job in `specs.py`. This keeps the generator a thin, deterministic extraction
step and puts every naming decision in one reviewable place per instrument.

## Consequences

- `streams_parsed.py` is regenerated by rerunning the CLI against an updated geometry file and
  committing the diff; nothing currently detects staleness against the latest geometry
  automatically. An earlier drift test was dropped because it depended on a geometry file not
  available in CI's Pooch cache, so it skipped in CI and gave false confidence rather than
  catching the realistic failure mode. A real check needs a stable geometry-file input pinned
  in Pooch first.
- `Stream` is a frozen dataclass that round-trips through any serialisation layer, so a future
  move to runtime or dynamically-distributed stream data would only change who populates
  `Instrument.streams`, not the shape consumers see.