ADR 0015: Disable support for broadcasting variances#

  • Status: accepted

  • Deciders: Jan-Lukas, Neil, Simon

  • Date: 2023-01-10

Context#

One of Scipp’s key features is propagation of uncertainties in operations. The mechanism was carried over from an equivalent mechanism in Mantid. Our recent publication [preprint submitted to Journal of Neutron Science] highlights a ubiquitous problem in this mechanism.

In neutron-scattering applications normalization terms typically have lower dimensionality than the data and are thus broadcast in operations. This leads to unhandled correlations in the normalized result. The effect is that uncertainties of normalization terms are strongly suppressed and effectively ignored. However, the user is not aware of this as the mechanism that Scipp provides seems to promise correct handling. This can lead to erroneous attribution of statistical significance in final results that scientists may publish.

We have considered issuing a warning instead of raising an exception. However, past experience shows that warnings tend to get ignored. Given the severity of the issue we thus do not consider this a solution.

Decision#

Raise an exception when implicitly or explicitly broadcasting variable elements that have variances.

Consequences#

Positive:#

  • Avoids hidden underestimation of uncertainties.

  • Avoid futile burning of CPU resources.

  • May help to push scientists to adopt improved approaches for handling uncertainties.

Negative:#

  • User action is required, as this will break most workflows.