Transforming data#

Overview#

Scipp can apply custom operations to the elements of one or more variables. This is essentially are more advanced version of std::transform.

Two alternatives are provided:

  • transform_in_place transforms its first argument.

  • transform returns a new Variable.

Both variants support:

  • Automatic broadcasting and alignment based on dimension labels. This does also include event data, and operations mixing events and dense data are supported.

  • Automatic propagation of uncertainties, provided that the user-provided operation is built from known existing operations.

  • Operations between different data types, and operations that produce outputs with a new data type.

  • Handling of units.

Basic usage#

Transform requires:

  • A list of types (or type-combinations) to support. Code is only instantiated for variables of those types.

  • A functor, typically a lambda or an “overloaded” lambda. This can also be used to pass special flags, e.g., for disabling code generation for arguments that have variances, generating a runtime failure instead.

Example 1#

Transform two variables with two type combinations:

  • a of type double and b of type float

  • a of type double and b of type double

Since + is defined for units::Unit the same lambda can be used for data and unit. If we needed to handle the unit manually (for example no operator which accepts it) we can use overloaded as shown in example 2.

This call to transform will add the two variables (or variable views) a and b and return a new variable.

auto var = transform<
    std::tuple<std::tuple<double, float>, std::tuple<double, double>>>(
    a, b, [](const auto &a_, const auto &b_) { return a_ + b_; });

Example 2#

This call to transform_in_place accepts two variables (or variable views) a and b. a is modified in-place, no new variable is returned. The example also has special unit handling using overloaded, to define an “overloaded” lambda.

transform_in_place<std::tuple<bool>>(
    a, b,
    overloaded{[](auto &a_, const auto &b_) { a &= b; },
               [](const units::Unit &a, const units::Unit &b) {
                 if (a != b)
                   throw std::runtime_error("Unit must match");
               }});

transform_in_place modifies its first argument. Note that the template argument std::tuple<bool> is equivalent to std::tuple<std::tuple<bool, bool>>, i.e., both arguments are require have element type bool.