.ipynb .pdf

repository open issue suggest edit

Scipp version

latest v0.10 v0.9 v0.8 v0.7 v0.6 v0.5 v0.4 v0.3

Related projects

scipp scippneutron ess

Contents

What’s new in scipp

This page highlights feature additions and discusses major changes from recent releases. For a full list of changes see the Release Notes.

import numpy as np
import scipp as sc

General

Get unique dimension using dim property

New in 0.9

The new dim property checks whether an object is 1-D, and returns the only dimension label. An exception is raised if the object is not 1-D.

Example:

x = sc.linspace(dim='x', start=0, stop=1, num=4)
x.dim

'x'

Logging support

New in 0.9

Scipp now provides a logger, and a pre-configured logging widget for Jupyter notebooks. See Logging.

Bound method equivalents to many free functions

New in 0.8

Many functions that have been available as free functions can now be used also as methods of variables and data arrays. See the documentation for individual classes for a full list.

Example:

var = sc.arange(dim="x", unit="m", start=0, stop=12)
var.sum()  # Previously sc.sum(var)

scipp.Variable (8 Bytes)

• • ()
    int64
    m
    66

    Values:
    array(66)

Note that sc.sum(var) will continue to be supported as well.

Indexing

Ellipsis

New in 0.8

Indexing with ellipsis (...) is now supported. This can be used, e.g., to replace data in an existing object without re-pointing the underlying reference to the object given on the right-hand side.

Example

var1 = sc.ones(dims=["x"], shape=[4])
var2 = var1 + var1
da = sc.DataArray(data=sc.zeros(dims=["x"], shape=[4]))
da.data = var1  # replace data variable
da.data[...] = var2  # assign to slice, copy into existing data variable
var1  # now holds values of var2

scipp.Variable (32 Bytes)

• • (x: 4)
    float64
    2.0, 2.0, 2.0, 2.0

    Values:
    array([2., 2., 2., 2.])

Changing var2 has no effect on da.data:

var2 += 2222.0
da

scipp.DataArray (32 Bytes)

• Dimensions:
  • x: 4
• Data: (1)
  • (x)
    float64
    2.0, 2.0, 2.0, 2.0

    Values:
    array([2., 2., 2., 2.])

Operations

Creation functions

New in 0.5

For convenience and similarity to numpy we added functions that create variables. Our intention is to fully replace the need to use sc.Variable directly, but at this point this has not been rolled out to our documentation pages.

Examples:

sc.array(dims=["x"], values=np.array([1, 2, 3]))

scipp.Variable (24 Bytes)

• • (x: 3)
    int64
    1, 2, 3

    Values:
    array([1, 2, 3])

sc.zeros(dims=["x"], shape=[3])

scipp.Variable (24 Bytes)

• • (x: 3)
    float64
    0.0, 0.0, 0.0

    Values:
    array([0., 0., 0.])

sc.scalar(17)

scipp.Variable (8 Bytes)

• • ()
    int64
    17

    Values:
    array(17)

All of these also take keyword arguments. Note that we can still support creating scalars by multiplying with a unit:

1.2 * sc.units.m

scipp.Variable (8 Bytes)

• • ()
    float64
    m
    1.2

    Values:
    array(1.2)

New in 0.7

More creation functions were added:

• Added zeros_like, ones_like, and empty_like.

• Added linspace, logspace, geomspace, and arange.

New in 0.8

More creation functions were added:

• Added full and full_like.

Unit conversion

New in 0.6

Conversions between different unit scales are now supported. to_unit provides conversion of variables between, e.g., mm and m.

New in 0.7

• to_unit can now avoid making a copy if the input already has the desired unit. This can be used as a cheap way to ensure inputs have expected units.

• to_unit now also works for binned data, converting the unit of the underlying events in the bins

New in 0.8

• to_unit now has a copy argument. By default, copy=True and to_unit makes a copy even if the input already has the desired unit. For a cheap way to ensure inputs have expected units use copy=False to avoid copies if possible.

Example:

var = sc.array(dims=["x"], unit="mm", values=[3.2, 5.4, 7.6])
m = sc.to_unit(var, "m")
m

scipp.Variable (24 Bytes)

• • (x: 3)
    float64
    m
    0.0, 0.01, 0.01

    Values:
    array([0.0032, 0.0054, 0.0076])

No copy is made if the input has the requested unit when we specify copy=False:

sc.to_unit(m, "m", copy=False)  # no copy

scipp.Variable (24 Bytes)

• • (x: 3)
    float64
    m
    0.0, 0.01, 0.01

    Values:
    array([0.0032, 0.0054, 0.0076])

Conversions also work for more specialized units such as electron-volt:

sc.to_unit(sc.scalar(1.0, unit="nJ"), unit="meV")

scipp.Variable (8 Bytes)

• • ()
    float64
    meV
    6241509074460.764

    Values:
    array(6.24150907e+12)

from_pandas and from_xarray

New in 0.8

• from_pandas for converting pandas.Dataframe to scipp.Dataset.

• from_xarray for converting xarray.DataArray or xarray.Dataset to scipp.DataAray or scipp.Dataset, respectively.

Both functions are available in the compat submodule.

Reduction operations

Internal precision in summation operations

New in 0.9

Reduction operations such as sum of single-precision (float32) data now use double-precision (float64) internally to reduce the effects of rounding errors.

Reductions over multiple inputs using reduce

New in 0.9

The new reduce function can be used for reduction operations that do not operate along a dimension of a scipp object but rather across a list or tuple of multiple scipp objects. The mechanism is a 2-step approach, with a syntasx similar to groupby:

a = sc.linspace(dim="x", start=0.0, stop=1.0, num=4)
b = sc.linspace(dim="x", start=0.2, stop=0.8, num=4)
c = sc.linspace(dim="x", start=0.2, stop=1.2, num=4)
sc.reduce([a, b, c]).sum()

scipp.Variable (32 Bytes)

• • (x: 4)
    float64
    0.4, 1.27, 2.13, 3.0

    Values:
    array([0.4       , 1.26666667, 2.13333333, 3.        ])

reducer = sc.reduce([a, b, c])
reducer.min()

scipp.Variable (32 Bytes)

• • (x: 4)
    float64
    0.0, 0.33, 0.6, 0.8

    Values:
    array([0.        , 0.33333333, 0.6       , 0.8       ])

reducer.max()

scipp.Variable (32 Bytes)

• • (x: 4)
    float64
    0.2, 0.53, 0.87, 1.2

    Values:
    array([0.2       , 0.53333333, 0.86666667, 1.2       ])

Shape operations

concat replacing concatenate

New in 0.9

concat is replacing concatenate (which is deprecated now and will be removed in 0.10). It supports a list of inputs rather than just 2 inputs.

a = sc.scalar(1.2)
b = sc.scalar(2.3)
c = sc.scalar(3.4)
sc.concat([a, b, c], "x")

scipp.Variable (24 Bytes)

• • (x: 3)
    float64
    1.2, 2.3, 3.4

    Values:
    array([1.2, 2.3, 3.4])

fold and flatten

New in 0.6

fold and flatten, which are similar to numpy.reshape, have been added. In contrast to reshape, fold and flatten support data arrays and handle also meta data such as coord, masks, and attrs.

New in 0.7

• fold now always returns views of data and all meta data instead of making deep copies.

• flatten also preserves reshaped data as a view, but unlike fold the same is not true for meta data in general, since it may require duplication in the flatten operation.

Example:

var = sc.ones(dims=["pixel"], shape=[100])
xy = sc.fold(var, dim="pixel", sizes={"x": 10, "y": 10})
xy = sc.DataArray(
    data=xy,
    coords={
        "x": sc.array(dims=["x"], values=np.arange(10)),
        "y": sc.array(dims=["y"], values=np.arange(10)),
    },
)
xy

scipp.DataArray (960 Bytes)

• Dimensions:
  • x: 10
  • y: 10
• Coordinates: (2)
  • x
    (x)
    int64
    0, 1, ..., 8, 9

    Values:
    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

  • y
    (y)
    int64
    0, 1, ..., 8, 9

    Values:
    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

• Data: (1)
  • (x, y)
    float64
    1.0, 1.0, ..., 1.0, 1.0

    Values:
    array([[1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
           [1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]])

Folding does not effect copies of either data or meta data, for example:

xy["y", 4] *= 0.0  # affects var (scipp-0.7 and higher)
var.plot()

The reverse of fold is flatten:

flat = sc.flatten(xy, to="pixel")
flat

scipp.DataArray (2.34 KB)

• Dimensions:
  • pixel: 100
• Coordinates: (2)
  • x
    (pixel)
    int64
    0, 0, ..., 9, 9

    Values:
    array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2,
           2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4,
           4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6,
           6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8,
           8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9])

  • y
    (pixel)
    int64
    0, 1, ..., 8, 9

    Values:
    array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1,
           2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3,
           4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5,
           6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7,
           8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

• Data: (1)
  • (pixel)
    float64
    1.0, 1.0, ..., 1.0, 1.0

    Values:
    array([1., 1., 1., 1., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 0., 1., 1.,
           1., 1., 1., 1., 1., 1., 1., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
           0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 0., 1., 1., 1., 1., 1., 1.,
           1., 1., 1., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 0., 1., 1., 1.,
           1., 1., 1., 1., 1., 1., 0., 1., 1., 1., 1., 1., 1., 1., 1., 1., 0.,
           1., 1., 1., 1., 1., 1., 1., 1., 1., 0., 1., 1., 1., 1., 1.])

Flattening does not effect a copy of data, but meta data may get copied if values need to be duplicated by the operation:

flat["pixel", 0] = 22  # modifies var (scipp-0.7 and higher)
var.plot()

Vectors and matrices

General

New in 0.7

Several improvements for working with (3-D position) vectors and (3-D rotation) matrices are part of this release:

• Creation functions were added:

  • vector (a single vector)

  • vectors (array of vectors)

  • matrix (a single matrix),

  • matrices (array of matrices).

• Direct creation and initialization of 2-D (or higher) arrays of matrices and vectors is now possible from numpy arrays.

• The values property now returns a numpy array with ndim+1 (vectors) or ndim+2 (matrices) axes, with the inner 1 (vectors) or 2 (matrices) axes corresponding to the vector or matrix axes.

• Vector or matrix elements can now be accessed and modified directly using the new fields property of variables. fields provides access to vector elements x, y, and z or matrix elements xx, xy, …, zz.

New in 0.8

The fields property can now be iterated and behaves similar to a dict with fixed keys.

sc.vector(value=[1, 2, 3])

scipp.Variable (24 Bytes)

• • ()
    vector_3_float64
    [1. 2. 3.]

    Values:
    array([1., 2., 3.])

vecs = sc.vectors(dims=["x"], unit="m", values=np.arange(12).reshape(4, 3))
vecs

scipp.Variable (96 Bytes)

• • (x: 4)
    vector_3_float64
    m
    [0. 1. 2.], [3. 4. 5.], [6. 7. 8.], [ 9. 10. 11.]

    Values:
    array([[ 0.,  1.,  2.],
           [ 3.,  4.,  5.],
           [ 6.,  7.,  8.],
           [ 9., 10., 11.]])

vecs.values

array([[ 0.,  1.,  2.],
       [ 3.,  4.,  5.],
       [ 6.,  7.,  8.],
       [ 9., 10., 11.]])

vecs.fields.y

scipp.Variable (32 Bytes out of 96 Bytes)

• • (x: 4)
    float64
    m
    1.0, 4.0, 7.0, 10.0

    Values:
    array([ 1.,  4.,  7., 10.])

vecs.fields.z += 0.666 * sc.units.m
vecs

scipp.Variable (96 Bytes)

• • (x: 4)
    vector_3_float64
    m
    [0. 1. 2.666], [3. 4. 5.666], [6. 7. 8.666], [ 9. 10. 11.666]

    Values:
    array([[ 0.   ,  1.   ,  2.666],
           [ 3.   ,  4.   ,  5.666],
           [ 6.   ,  7.   ,  8.666],
           [ 9.   , 10.   , 11.666]])

New in 0.8

The cross function to compute the cross-product of vectors as added.

sc.cross(vecs, vecs["x", 0])

scipp.Variable (96 Bytes)

• • (x: 4)
    vector_3_float64
    m^2
    [0. 0. 0.], [ 4.998 -7.998 3. ], [ 9.996 -15.996 6. ], [ 14.994 -23.994 9. ]

    Values:
    array([[  0.   ,   0.   ,   0.   ],
           [  4.998,  -7.998,   3.   ],
           [  9.996, -15.996,   6.   ],
           [ 14.994, -23.994,   9.   ]])

scipp.spatial.transform

New in 0.8

The scipp.spatial.transform (in the style of scipy.spatial.transform) submodule was added. This now provides:

• from_rotvec to create rotation matrices from rotation vectors.

• as_rotvec to convert rotation matrices into rotation vectors.

As an example, the following creates a rotation matrix for rotation around the x-axis by 30 degrees:

from scipp.spatial.transform import from_rotvec

rot = from_rotvec(sc.vector(value=[30.0, 0, 0], unit="deg"))
rot

scipp.Variable (72 Bytes)

• • ()
    matrix_3_float64
    [[ 1. 0. 0. ] [ 0. 0.8660254 -0.5 ] [ 0. 0.5 0.8660254]]

    Values:
    array([[ 1.       ,  0.       ,  0.       ],
           [ 0.       ,  0.8660254, -0.5      ],
           [ 0.       ,  0.5      ,  0.8660254]])

Coordinate transformations

New in 0.8

The transform_coords function has been added (also available as method of data arrays and datasets). It is a tool for transforming one or more input coordinates into one or more output coordinates. It automatically handles:

• Renaming of dimensions, if dimension-coordinates are transformed.

• Change of coordinates to attributes to avoid interference of coordinates consumed by the transformation in follow-up operations.

• Conversion of event-coordinates of binned data, if present.

See Coordinate transformations for a full description.

Physical constants

New in 0.8

The scipp.constants (in the style of scipy.constants) submodule was added, providing physical constants from CODATA 2018. For full details see the module’s documentation.

Examples:

from scipp.constants import hbar, m_e, physical_constants

hbar

scipp.Variable (8 Bytes)

• • ()
    float64
    J*s
    1.0545718176461565e-34

    Values:
    array(1.05457182e-34)

m_e

scipp.Variable (8 Bytes)

• • ()
    float64
    kg
    9.1093837015e-31

    Values:
    array(9.1093837e-31)

physical_constants("speed of light in vacuum")

scipp.Variable (8 Bytes)

• • ()
    float64
    m/s
    299792458.0

    Values:
    array(2.99792458e+08)

physical_constants("neutron mass", with_variance=True)

scipp.Variable (16 Bytes)

• • ()
    float64
    kg
    1.67492749804e-27

    σ = 9.5e-37

    Values:
    array(1.6749275e-27)
    
    Variances (σ²):
    array(9.025e-73)

Plotting

New in 0.7

• Plotting supports redraw() method for updating existing plots with new data, without recreating the plot.

New in 0.8

• Plotting 1-D binned (event) data is now supported.

Binned data

Buffer and meta data access

New in 0.7

• The internal buffer holding the “events” underlying binned data can now be accessed directly using the new events property. Update: This is deprecated as of 0.8.2.

• HTML view now works for binned meta data access such as binned.bins.coords['time']

New in 0.8

The mean of bins can now be computed using binned.bins.mean(). This should general be used instead of binned.bins.sum() the if dtype is not “summable”, i.e., typically anything that is not of unit “counts”.

Consider the following example, representing a time series of temperature measurements on an x-y plane:

import numpy as np

N = int(800)
data = sc.DataArray(
    data=sc.Variable(dims=["time"], values=100 + np.random.rand(N) * 10, unit="K"),
    coords={
        "x": sc.Variable(dims=["time"], unit="m", values=np.random.rand(N)),
        "y": sc.Variable(dims=["time"], unit="m", values=np.random.rand(N)),
        "time": sc.Variable(
            dims=["time"], values=(10000 * np.random.rand(N)).astype("datetime64[s]")
        ),
    },
)
binned = sc.bin(
    data,
    edges=[
        sc.linspace(dim="x", unit="m", start=0.0, stop=1.0, num=5),
        sc.linspace(dim="y", unit="m", start=0.0, stop=1.0, num=5),
    ],
)
binned

scipp.DataArray (25.33 KB)

• Dimensions:
  • x: 4
  • y: 4
• Coordinates: (2)
  • x
    (x [bin-edge])
    float64
    m
    0.0, 0.25, 0.5, 0.75, 1.0

    Values:
    array([0.  , 0.25, 0.5 , 0.75, 1.  ])

  • y
    (y [bin-edge])
    float64
    m
    0.0, 0.25, 0.5, 0.75, 1.0

    Values:
    array([0.  , 0.25, 0.5 , 0.75, 1.  ])

• Data: (1)
  • (x, y)
    DataArrayView
    binned data [len=53, len=37, ..., len=54, len=50]

    Values:
    [<scipp.DataArray>
    Dimensions: Sizes[time:53, ]
    Coordinates:
      time                    datetime64              [s]  (time)  [1970-01-01T00:42:43, 1970-01-01T00:16:04, 1970-01-01T00:22:33, 1970-01-01T00:29:21, ..., 1970-01-01T02:05:02, 1970-01-01T02:07:56, 1970-01-01T02:07:37, 1970-01-01T00:45:42]
      x                         float64              [m]  (time)  [0.232669, 0.033015, 0.107412, 0.085357, ..., 0.003633, 0.053488, 0.181451, 0.191665]
      y                         float64              [m]  (time)  [0.067110, 0.031307, 0.211551, 0.192240, ..., 0.147991, 0.085272, 0.054944, 0.016516]
    Data:
                                float64              [K]  (time)  [101.257779, 108.109635, 105.110120, 102.314163, ..., 100.924175, 100.110019, 108.949049, 101.080443]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:37, ]
    Coordinates:
      time                    datetime64              [s]  (time)  [1970-01-01T01:27:36, 1970-01-01T02:03:09, 1970-01-01T00:14:11, 1970-01-01T01:12:57, ..., 1970-01-01T00:01:54, 1970-01-01T00:35:26, 1970-01-01T02:32:19, 1970-01-01T01:20:56]
      x                         float64              [m]  (time)  [0.234093, 0.097141, 0.004677, 0.110673, ..., 0.202910, 0.042513, 0.040133, 0.167972]
      y                         float64              [m]  (time)  [0.290356, 0.393585, 0.431111, 0.383108, ..., 0.385532, 0.412372, 0.344481, 0.410825]
    Data:
                                float64              [K]  (time)  [105.648075, 103.547795, 101.610305, 102.312072, ..., 106.607121, 109.267737, 103.831491, 109.347921]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:60, ]
    Coordinates:
      time                    datetime64              [s]  (time)  [1970-01-01T02:37:08, 1970-01-01T00:38:42, 1970-01-01T01:19:02, 1970-01-01T00:03:46, ..., 1970-01-01T01:38:59, 1970-01-01T00:59:15, 1970-01-01T02:14:26, 1970-01-01T01:29:23]
      x                         float64              [m]  (time)  [0.035218, 0.068431, 0.016104, 0.150130, ..., 0.099295, 0.137601, 0.138658, 0.066025]
      y                         float64              [m]  (time)  [0.674263, 0.734628, 0.506059, 0.570593, ..., 0.558980, 0.642139, 0.621928, 0.524412]
    Data:
                                float64              [K]  (time)  [102.753980, 102.705576, 109.216182, 100.064331, ..., 103.316483, 102.802662, 104.139007, 104.344144]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:54, ]
    Coordinates:
      time                    datetime64              [s]  (time)  [1970-01-01T00:59:41, 1970-01-01T00:25:54, 1970-01-01T00:58:03, 1970-01-01T01:16:48, ..., 1970-01-01T02:05:09, 1970-01-01T01:13:11, 1970-01-01T02:32:40, 1970-01-01T02:32:30]
      x                         float64              [m]  (time)  [0.035298, 0.034438, 0.226005, 0.134065, ..., 0.044121, 0.115623, 0.221210, 0.146107]
      y                         float64              [m]  (time)  [0.972113, 0.937694, 0.964167, 0.889252, ..., 0.764767, 0.855167, 0.967084, 0.845281]
    Data:
                                float64              [K]  (time)  [105.882235, 102.573285, 103.975585, 100.534760, ..., 100.395753, 103.598733, 104.547934, 106.726166]
    
    , ..., <scipp.DataArray>
    Dimensions: Sizes[time:38, ]
    Coordinates:
      time                    datetime64              [s]  (time)  [1970-01-01T00:42:04, 1970-01-01T00:29:44, 1970-01-01T01:15:50, 1970-01-01T02:26:38, ..., 1970-01-01T00:55:20, 1970-01-01T00:48:07, 1970-01-01T00:23:56, 1970-01-01T00:37:39]
      x                         float64              [m]  (time)  [0.831138, 0.965937, 0.843333, 0.841413, ..., 0.893211, 0.837393, 0.999951, 0.790882]
      y                         float64              [m]  (time)  [0.161395, 0.239136, 0.132679, 0.206543, ..., 0.074474, 0.095151, 0.028776, 0.213673]
    Data:
                                float64              [K]  (time)  [104.079837, 105.815366, 103.243311, 106.253583, ..., 103.157748, 107.588624, 102.261760, 108.772920]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:49, ]
    Coordinates:
      time                    datetime64              [s]  (time)  [1970-01-01T00:48:49, 1970-01-01T01:49:36, 1970-01-01T02:28:58, 1970-01-01T00:35:33, ..., 1970-01-01T02:41:00, 1970-01-01T00:04:25, 1970-01-01T01:49:38, 1970-01-01T01:57:10]
      x                         float64              [m]  (time)  [0.960071, 0.788760, 0.824544, 0.813434, ..., 0.859143, 0.987661, 0.769742, 0.975881]
      y                         float64              [m]  (time)  [0.351715, 0.445431, 0.378798, 0.300279, ..., 0.389268, 0.278537, 0.396026, 0.330684]
    Data:
                                float64              [K]  (time)  [102.960521, 104.981683, 109.215668, 106.007120, ..., 100.011854, 108.600377, 106.998447, 103.132851]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:54, ]
    Coordinates:
      time                    datetime64              [s]  (time)  [1970-01-01T00:35:41, 1970-01-01T00:39:28, 1970-01-01T00:02:40, 1970-01-01T00:54:25, ..., 1970-01-01T00:22:14, 1970-01-01T02:04:56, 1970-01-01T00:10:18, 1970-01-01T01:36:35]
      x                         float64              [m]  (time)  [0.771017, 0.875914, 0.874725, 0.992106, ..., 0.900347, 0.816283, 0.988733, 0.766969]
      y                         float64              [m]  (time)  [0.747732, 0.644240, 0.555238, 0.676909, ..., 0.560860, 0.691540, 0.660530, 0.654508]
    Data:
                                float64              [K]  (time)  [100.268974, 107.978814, 108.805276, 102.387481, ..., 100.839271, 100.201653, 104.514647, 104.445200]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:50, ]
    Coordinates:
      time                    datetime64              [s]  (time)  [1970-01-01T00:53:35, 1970-01-01T02:41:57, 1970-01-01T00:47:46, 1970-01-01T02:34:08, ..., 1970-01-01T02:09:12, 1970-01-01T00:35:10, 1970-01-01T01:15:57, 1970-01-01T00:48:06]
      x                         float64              [m]  (time)  [0.833008, 0.813439, 0.966626, 0.881992, ..., 0.975862, 0.782134, 0.756033, 0.824134]
      y                         float64              [m]  (time)  [0.959362, 0.787553, 0.781100, 0.880398, ..., 0.843950, 0.930202, 0.880165, 0.862639]
    Data:
                                float64              [K]  (time)  [102.690277, 103.540084, 100.524541, 100.957069, ..., 105.128358, 102.783720, 104.984640, 106.093326]
    
    ]

sc.show(binned)

To allow for this, the bins property provides properties data, coords, masks, and attrs of the bins that behave like the properties of a data array while retaining the binned structure. That is, it can be used for computation involving information available on a per-bin basis:

binned.bins.coords["time"]

scipp.Variable (6.50 KB)

• • (x: 4, y: 4)
    VariableView
    binned data [len=53, len=37, ..., len=54, len=50]

    Values:
    [<scipp.Variable> (time: 53)  datetime64              [s]  [1970-01-01T00:42:43, 1970-01-01T00:16:04, 1970-01-01T00:22:33, 1970-01-01T00:29:21, ..., 1970-01-01T02:05:02, 1970-01-01T02:07:56, 1970-01-01T02:07:37, 1970-01-01T00:45:42], <scipp.Variable> (time: 37)  datetime64              [s]  [1970-01-01T01:27:36, 1970-01-01T02:03:09, 1970-01-01T00:14:11, 1970-01-01T01:12:57, ..., 1970-01-01T00:01:54, 1970-01-01T00:35:26, 1970-01-01T02:32:19, 1970-01-01T01:20:56], <scipp.Variable> (time: 60)  datetime64              [s]  [1970-01-01T02:37:08, 1970-01-01T00:38:42, 1970-01-01T01:19:02, 1970-01-01T00:03:46, ..., 1970-01-01T01:38:59, 1970-01-01T00:59:15, 1970-01-01T02:14:26, 1970-01-01T01:29:23], <scipp.Variable> (time: 54)  datetime64              [s]  [1970-01-01T00:59:41, 1970-01-01T00:25:54, 1970-01-01T00:58:03, 1970-01-01T01:16:48, ..., 1970-01-01T02:05:09, 1970-01-01T01:13:11, 1970-01-01T02:32:40, 1970-01-01T02:32:30], ..., <scipp.Variable> (time: 38)  datetime64              [s]  [1970-01-01T00:42:04, 1970-01-01T00:29:44, 1970-01-01T01:15:50, 1970-01-01T02:26:38, ..., 1970-01-01T00:55:20, 1970-01-01T00:48:07, 1970-01-01T00:23:56, 1970-01-01T00:37:39], <scipp.Variable> (time: 49)  datetime64              [s]  [1970-01-01T00:48:49, 1970-01-01T01:49:36, 1970-01-01T02:28:58, 1970-01-01T00:35:33, ..., 1970-01-01T02:41:00, 1970-01-01T00:04:25, 1970-01-01T01:49:38, 1970-01-01T01:57:10], <scipp.Variable> (time: 54)  datetime64              [s]  [1970-01-01T00:35:41, 1970-01-01T00:39:28, 1970-01-01T00:02:40, 1970-01-01T00:54:25, ..., 1970-01-01T00:22:14, 1970-01-01T02:04:56, 1970-01-01T00:10:18, 1970-01-01T01:36:35], <scipp.Variable> (time: 50)  datetime64              [s]  [1970-01-01T00:53:35, 1970-01-01T02:41:57, 1970-01-01T00:47:46, 1970-01-01T02:34:08, ..., 1970-01-01T02:09:12, 1970-01-01T00:35:10, 1970-01-01T01:15:57, 1970-01-01T00:48:06]]

sc.show(binned.bins.coords["time"])

We can use this in our example to correct for an hypothetical clock error that depends on the x-y bin:

clock_correction = sc.array(
    dims=["x", "y"], unit="s", values=(100 * np.random.rand(4, 4)).astype("int64")
)
clock_correction

scipp.Variable (128 Bytes)

• • (x: 4, y: 4)
    int64
    s
    59, 24, ..., 48, 32

    Values:
    array([[59, 24, 66, 18],
           [28, 44, 64, 99],
           [52, 20, 71, 85],
           [91, 83, 48, 32]])

binned.bins.coords["time"] += clock_correction

The properties can also be used to add or delete meta data entries:

del binned.bins.coords["x"]

Broadcasting dense variables to binned variables using bins_like

New in 0.9

• Added bins_like, for broadcasting dense variables to binned variables, e.g., for converting bin coordinates into event coordinates.

temperature = sc.array(dims=['x'], unit='K', values=[3.,4.,5.,6.])
binned.bins.coords['temperature'] = sc.bins_like(binned, fill_value=temperature)
binned

scipp.DataArray (25.33 KB)

• Dimensions:
  • x: 4
  • y: 4
• Coordinates: (2)
  • x
    (x [bin-edge])
    float64
    m
    0.0, 0.25, 0.5, 0.75, 1.0

    Values:
    array([0.  , 0.25, 0.5 , 0.75, 1.  ])

  • y
    (y [bin-edge])
    float64
    m
    0.0, 0.25, 0.5, 0.75, 1.0

    Values:
    array([0.  , 0.25, 0.5 , 0.75, 1.  ])

• Data: (1)
  • (x, y)
    DataArrayView
    binned data [len=53, len=37, ..., len=54, len=50]

    Values:
    [<scipp.DataArray>
    Dimensions: Sizes[time:53, ]
    Coordinates:
      temperature               float64              [K]  (time)  [3.000000, 3.000000, 3.000000, 3.000000, ..., 3.000000, 3.000000, 3.000000, 3.000000]
      time                    datetime64              [s]  (time)  [1970-01-01T00:43:42, 1970-01-01T00:17:03, 1970-01-01T00:23:32, 1970-01-01T00:30:20, ..., 1970-01-01T02:06:01, 1970-01-01T02:08:55, 1970-01-01T02:08:36, 1970-01-01T00:46:41]
      y                         float64              [m]  (time)  [0.067110, 0.031307, 0.211551, 0.192240, ..., 0.147991, 0.085272, 0.054944, 0.016516]
    Data:
                                float64              [K]  (time)  [101.257779, 108.109635, 105.110120, 102.314163, ..., 100.924175, 100.110019, 108.949049, 101.080443]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:37, ]
    Coordinates:
      temperature               float64              [K]  (time)  [3.000000, 3.000000, 3.000000, 3.000000, ..., 3.000000, 3.000000, 3.000000, 3.000000]
      time                    datetime64              [s]  (time)  [1970-01-01T01:28:00, 1970-01-01T02:03:33, 1970-01-01T00:14:35, 1970-01-01T01:13:21, ..., 1970-01-01T00:02:18, 1970-01-01T00:35:50, 1970-01-01T02:32:43, 1970-01-01T01:21:20]
      y                         float64              [m]  (time)  [0.290356, 0.393585, 0.431111, 0.383108, ..., 0.385532, 0.412372, 0.344481, 0.410825]
    Data:
                                float64              [K]  (time)  [105.648075, 103.547795, 101.610305, 102.312072, ..., 106.607121, 109.267737, 103.831491, 109.347921]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:60, ]
    Coordinates:
      temperature               float64              [K]  (time)  [3.000000, 3.000000, 3.000000, 3.000000, ..., 3.000000, 3.000000, 3.000000, 3.000000]
      time                    datetime64              [s]  (time)  [1970-01-01T02:38:14, 1970-01-01T00:39:48, 1970-01-01T01:20:08, 1970-01-01T00:04:52, ..., 1970-01-01T01:40:05, 1970-01-01T01:00:21, 1970-01-01T02:15:32, 1970-01-01T01:30:29]
      y                         float64              [m]  (time)  [0.674263, 0.734628, 0.506059, 0.570593, ..., 0.558980, 0.642139, 0.621928, 0.524412]
    Data:
                                float64              [K]  (time)  [102.753980, 102.705576, 109.216182, 100.064331, ..., 103.316483, 102.802662, 104.139007, 104.344144]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:54, ]
    Coordinates:
      temperature               float64              [K]  (time)  [3.000000, 3.000000, 3.000000, 3.000000, ..., 3.000000, 3.000000, 3.000000, 3.000000]
      time                    datetime64              [s]  (time)  [1970-01-01T00:59:59, 1970-01-01T00:26:12, 1970-01-01T00:58:21, 1970-01-01T01:17:06, ..., 1970-01-01T02:05:27, 1970-01-01T01:13:29, 1970-01-01T02:32:58, 1970-01-01T02:32:48]
      y                         float64              [m]  (time)  [0.972113, 0.937694, 0.964167, 0.889252, ..., 0.764767, 0.855167, 0.967084, 0.845281]
    Data:
                                float64              [K]  (time)  [105.882235, 102.573285, 103.975585, 100.534760, ..., 100.395753, 103.598733, 104.547934, 106.726166]
    
    , ..., <scipp.DataArray>
    Dimensions: Sizes[time:38, ]
    Coordinates:
      temperature               float64              [K]  (time)  [6.000000, 6.000000, 6.000000, 6.000000, ..., 6.000000, 6.000000, 6.000000, 6.000000]
      time                    datetime64              [s]  (time)  [1970-01-01T00:43:35, 1970-01-01T00:31:15, 1970-01-01T01:17:21, 1970-01-01T02:28:09, ..., 1970-01-01T00:56:51, 1970-01-01T00:49:38, 1970-01-01T00:25:27, 1970-01-01T00:39:10]
      y                         float64              [m]  (time)  [0.161395, 0.239136, 0.132679, 0.206543, ..., 0.074474, 0.095151, 0.028776, 0.213673]
    Data:
                                float64              [K]  (time)  [104.079837, 105.815366, 103.243311, 106.253583, ..., 103.157748, 107.588624, 102.261760, 108.772920]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:49, ]
    Coordinates:
      temperature               float64              [K]  (time)  [6.000000, 6.000000, 6.000000, 6.000000, ..., 6.000000, 6.000000, 6.000000, 6.000000]
      time                    datetime64              [s]  (time)  [1970-01-01T00:50:12, 1970-01-01T01:50:59, 1970-01-01T02:30:21, 1970-01-01T00:36:56, ..., 1970-01-01T02:42:23, 1970-01-01T00:05:48, 1970-01-01T01:51:01, 1970-01-01T01:58:33]
      y                         float64              [m]  (time)  [0.351715, 0.445431, 0.378798, 0.300279, ..., 0.389268, 0.278537, 0.396026, 0.330684]
    Data:
                                float64              [K]  (time)  [102.960521, 104.981683, 109.215668, 106.007120, ..., 100.011854, 108.600377, 106.998447, 103.132851]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:54, ]
    Coordinates:
      temperature               float64              [K]  (time)  [6.000000, 6.000000, 6.000000, 6.000000, ..., 6.000000, 6.000000, 6.000000, 6.000000]
      time                    datetime64              [s]  (time)  [1970-01-01T00:36:29, 1970-01-01T00:40:16, 1970-01-01T00:03:28, 1970-01-01T00:55:13, ..., 1970-01-01T00:23:02, 1970-01-01T02:05:44, 1970-01-01T00:11:06, 1970-01-01T01:37:23]
      y                         float64              [m]  (time)  [0.747732, 0.644240, 0.555238, 0.676909, ..., 0.560860, 0.691540, 0.660530, 0.654508]
    Data:
                                float64              [K]  (time)  [100.268974, 107.978814, 108.805276, 102.387481, ..., 100.839271, 100.201653, 104.514647, 104.445200]
    
    , <scipp.DataArray>
    Dimensions: Sizes[time:50, ]
    Coordinates:
      temperature               float64              [K]  (time)  [6.000000, 6.000000, 6.000000, 6.000000, ..., 6.000000, 6.000000, 6.000000, 6.000000]
      time                    datetime64              [s]  (time)  [1970-01-01T00:54:07, 1970-01-01T02:42:29, 1970-01-01T00:48:18, 1970-01-01T02:34:40, ..., 1970-01-01T02:09:44, 1970-01-01T00:35:42, 1970-01-01T01:16:29, 1970-01-01T00:48:38]
      y                         float64              [m]  (time)  [0.959362, 0.787553, 0.781100, 0.880398, ..., 0.843950, 0.930202, 0.880165, 0.862639]
    Data:
                                float64              [K]  (time)  [102.690277, 103.540084, 100.524541, 100.957069, ..., 105.128358, 102.783720, 104.984640, 106.093326]
    
    ]

Performance

New in 0.7

• sort is now considerably faster for data with more rows.

• reduction operations such as sum and mean are now also multi-threaded and thus considerably faster.

New in 0.9

• sc.lookup(histogram, dim)[var] is now faster if histogram is very long and is integer-valued. This is relevant in a number of event-filtering operations.

previous

About scipp

next

Roadmap