⚠️ You are viewing the documentation for an old version of scipp. Switch to latest version. ⚠️

Contents

Multi-dimensional datasets

Contents

Multi-dimensional datasets#

This is the continuation of 1-D datasets and tables.

Interaction with `numpy`#

Variable in a dataset are exposed in a numpy-compatible buffer format, so we can directly hand them to numpy functions:

[16]:

d['alice'].values = np.sin(d['alice'].values)

In contrast to the 1-D case considered earlier, the values are now a multi-dimensional array:

[17]:

d['alice'].values

[17]:

array([[[ 0.42637374,  0.10327389, -0.44985934, -0.88038432],
        [ 0.7438664 ,  0.17532136, -0.05720567, -0.18986241],
        [ 0.90797444, -0.25598074, -0.88544125, -0.08941917],
        [ 0.59026883, -0.06612826, -0.25000682, -0.55862891],
        [ 0.37935665,  0.05381862, -0.10919305, -0.68566015],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.69089017,  0.54395272,  0.42086657, -0.2285062 ],
        [ 0.4358927 , -0.18220115, -0.33869733, -0.53986735],
        [ 0.53675249, -0.16387416, -0.69313129,  0.43153279],
        [ 0.412811  ,  0.02248492, -0.91745494,  0.02426978]],

       [[ 0.60822091,  0.94700502, -0.57186694, -0.42043153],
        [ 0.63171049, -0.2048052 , -0.81802073, -0.12588752],
        [ 0.16884229,  0.17124439, -0.61629894, -0.53529045],
        [ 0.62146936,  0.35587082,  0.23699384, -0.23102101],
        [ 0.09690911,  0.31040083, -0.30556742, -0.58491692],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [-0.5291402 ,  0.34977298, -0.0738278 ,  0.12647207],
        [-0.01547088,  0.85526105, -0.79431865, -0.08600907],
        [ 0.43580629,  0.97630143, -0.28574301,  0.51276934],
        [-0.19543378,  0.21541389,  0.4941342 , -0.79590344]],

       [[ 0.33797665, -0.50979422,  0.10939777, -0.045666  ],
        [-0.5995035 , -0.39871657,  0.13349475, -0.6427325 ],
        [ 0.06626824, -0.53488104,  0.51191752, -0.61829577],
        [-0.73478726, -0.46075997,  0.28352496, -0.86969898],
        [ 0.57412327, -0.58627161,  0.31672365, -0.46725482],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.65113145, -0.23543109,  0.39847087, -0.50228215],
        [-0.67911339, -0.35834221,  0.34418667, -0.264974  ],
        [-0.03505913, -0.43397149,  0.02942011, -0.2841073 ],
        [-0.44874638, -0.03089734,  0.7310724 ,  0.04051578]],

       [[-0.88176196,  0.21067737,  0.25056534,  0.43752977],
        [-0.29116603,  0.33038578, -0.68593467,  0.69949335],
        [-0.32332367,  0.60500225, -0.05685078, -0.37651149],
        [-0.18824931,  0.59702078, -0.90326811,  0.40942399],
        [ 0.35354066, -0.5212613 , -0.00481993, -0.02742185],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [-0.09142968, -0.34584542, -0.11813388,  0.61941444],
        [-0.57159181,  0.10059659,  0.07128768, -0.07743276],
        [-0.23563404, -0.09456845, -0.29095137,  0.52609743],
        [ 0.45929126, -0.09670093, -0.47132729, -0.20660719]],

       [[-0.08195487, -0.40929836,  0.03669962, -0.02665829],
        [ 0.36434961, -0.1727823 ,  0.16494986, -0.2736747 ],
        [-0.56271313, -0.21102814,  0.54966297,  0.68565585],
        [ 0.26656663, -0.21798012,  0.54978998, -0.09725091],
        [ 0.44257433, -0.69969835,  0.39112775,  0.07540385],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.38951561, -0.64078397, -0.42682643,  0.38872138],
        [ 0.30051157, -0.98132433,  0.37603807,  0.48780999],
        [ 0.22621949, -0.59712481,  0.17094629, -0.09062409],
        [-0.49644188, -0.20056411, -0.28538918,  0.39819463]],

       [[-0.7780934 ,  0.73122261, -0.10183881, -0.66017818],
        [-0.63050428, -0.10081237, -0.86642754, -0.77899656],
        [-0.43545637, -0.34046326, -0.43991965, -0.72585941],
        [-0.82209581, -0.4451455 , -0.47176618, -0.05054229],
        [-0.69492226,  0.56308784, -0.97734881, -0.14507503],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [-0.15485534, -0.12753564, -0.70515701, -0.26524729],
        [-0.84314733,  0.05896058, -0.79499429, -0.10980846],
        [-0.68935727, -0.31345654, -0.65605383, -0.96385337],
        [-0.68396304,  0.20109699, -0.5162077 , -0.83079834]],

       [[-0.28878352,  0.56696835,  0.68434013, -0.93098205],
        [-0.19049392,  0.39346737,  0.93150499, -0.29557839],
        [-0.89807282,  0.28418337,  0.65882319, -0.56440972],
        [-0.54360687, -0.33921714,  0.69108434,  0.31710534],
        [-0.81820029,  0.1209076 ,  0.99454631, -0.84762395],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [-0.64498789, -0.04735861,  0.39438537,  0.20826328],
        [-0.54050879,  0.91486932,  0.50599845, -0.56335042],
        [-0.97956211,  0.23559222,  0.788313  , -0.80885451],
        [-0.69437659,  0.17122095,  0.76688662, -0.04873422]],

       [[-0.07594797, -0.61115551, -0.09810116,  0.0462285 ],
        [-0.45695057, -0.10922783, -0.41639501,  0.01915997],
        [-0.68884201,  0.34372091, -0.43095998,  0.20542698],
        [-0.72168111,  0.12228407,  0.04140728, -0.23670664],
        [-0.65427259, -0.32641565, -0.75440322,  0.2259221 ],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [-0.7856857 , -0.22993075,  0.03150649,  0.06857374],
        [ 0.26112444,  0.74646398, -0.64924026, -0.56880373],
        [-0.16606625, -0.48606978, -0.35612894,  0.1432361 ],
        [-0.28982828, -0.27603608, -0.24213712, -0.12080088]]])

Exercise 3#

Use sc.mean to compute the mean of the data for Alice along the Z dimension.
Do the same with numpy, what are the complications you encounter, that are not present when using the dataset?

Solution 3#

[18]:

help(sc.mean)

Help on function mean in module scipp.core.reduction:

mean(x: 'VariableLikeType', dim: 'Optional[str]' = None) -> 'VariableLikeType'
    Arithmetic mean of elements in the input.

    If the input has variances, the variances stored in the output are based on
    the "standard deviation of the mean", i.e.,
    :math:`\sigma_{mean} = \sigma / \sqrt{N}`.
    :math:`N` is the length of the input dimension.
    :math:`\sigma` is estimated as the average of the standard deviations of
    the input elements along that dimension.

    See :py:func:`scipp.sum` on how rounding errors for float32 inputs are handled.

    Parameters
    ----------
    x: scipp.typing.VariableLike
        Input data.
    dim:
        Dimension along which to calculate the mean. If not
        given, the mean over all dimensions is calculated.

    Returns
    -------
    : Same type as x
        The mean of the input values.

    See Also
    --------
    scipp.nanmean:
        Ignore NaN's when calculating the mean.

[19]:

mean = sc.mean(d['alice'], 'z')

When using numpy to compute the mean: - We must remember (or lookup) which dimension corresponds to the Z dimensions. - We need a separate call for values and variances. - We need to manually scale the variance with the inverse of the number of data points to get the variance of the mean (standard deviation of the mean scales with 1/sqrt(N)).

[20]:

np_values = np.mean(d['alice'].values, axis=0)
np_variances = np.mean(d['alice'].variances, axis=0)
np_variances /= d['alice'].shape[0]

[21]:

print(mean.values)
print(mean.variances)
print(np_values)
print(np_variances)

[[-0.0917463   0.12861239 -0.01758292 -0.31006776]
 [-0.05358648 -0.01089622 -0.20175425 -0.19850984]
 [-0.22066538  0.00772472 -0.08863336 -0.2523379 ]
 [-0.19151444 -0.05675692  0.02221991 -0.16466493]
 [-0.04011139 -0.135679   -0.05611684 -0.30707835]
 [ 0.          0.          0.          0.        ]
 [-0.0593202  -0.09164497 -0.00983948  0.05192616]
 [-0.20653794  0.14428548 -0.15996746 -0.21530448]
 [-0.11336257 -0.10964645 -0.16166613 -0.06672545]
 [-0.24208596  0.00075229 -0.05505287 -0.19248298]]
[[0.04644504 0.05349858 0.05173536 0.05077948]
 [0.05315813 0.05150951 0.04899382 0.05021359]
 [0.05038203 0.05199407 0.05010011 0.05329671]
 [0.047096   0.05077632 0.05053977 0.05447735]
 [0.05190424 0.05110722 0.04776057 0.0500711 ]
 [0.04799832 0.0550015  0.04811343 0.05214359]
 [0.05138138 0.05488406 0.05017598 0.05212902]
 [0.05179721 0.05276095 0.04726025 0.04855698]
 [0.04940957 0.0543734  0.04663385 0.04948921]
 [0.04966293 0.05073676 0.05543044 0.05112892]]
[[-0.0917463   0.12861239 -0.01758292 -0.31006776]
 [-0.05358648 -0.01089622 -0.20175425 -0.19850984]
 [-0.22066538  0.00772472 -0.08863336 -0.2523379 ]
 [-0.19151444 -0.05675692  0.02221991 -0.16466493]
 [-0.04011139 -0.135679   -0.05611684 -0.30707835]
 [ 0.          0.          0.          0.        ]
 [-0.0593202  -0.09164497 -0.00983948  0.05192616]
 [-0.20653794  0.14428548 -0.15996746 -0.21530448]
 [-0.11336257 -0.10964645 -0.16166613 -0.06672545]
 [-0.24208596  0.00075229 -0.05505287 -0.19248298]]
[[0.04644504 0.05349858 0.05173536 0.05077948]
 [0.05315813 0.05150951 0.04899382 0.05021359]
 [0.05038203 0.05199407 0.05010011 0.05329671]
 [0.047096   0.05077632 0.05053977 0.05447735]
 [0.05190424 0.05110722 0.04776057 0.0500711 ]
 [0.04799832 0.0550015  0.04811343 0.05214359]
 [0.05138138 0.05488406 0.05017598 0.05212902]
 [0.05179721 0.05276095 0.04726025 0.04855698]
 [0.04940957 0.0543734  0.04663385 0.04948921]
 [0.04966293 0.05073676 0.05543044 0.05112892]]