⚠️ You are viewing the documentation for an old version of scipp. Switch to latest version. ⚠️

Contents

Multi-dimensional datasets

Contents

Multi-dimensional datasets#

This is the continuation of 1-D datasets and tables.

Interaction with `numpy`#

Variable in a dataset are exposed in a numpy-compatible buffer format, so we can directly hand them to numpy functions:

[16]:

d['alice'].values = np.sin(d['alice'].values)

In contrast to the 1-D case considered earlier, the values are now a multi-dimensional array:

[17]:

d['alice'].values

[17]:

array([[[ 0.26654534, -0.74669345, -0.48054085,  0.23993719],
        [ 0.15237631, -0.56568349, -0.76204725,  0.46719934],
        [-0.21080287, -0.9227189 , -0.35388102,  0.4607885 ],
        [ 0.24971565, -0.67497864,  0.19766596,  0.19360971],
        [-0.02692691, -0.78412031, -0.4256413 ,  0.15320748],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.13747701, -0.11518359,  0.47205546,  0.15326478],
        [-0.08580829,  0.04594232, -0.27692971,  0.19969754],
        [-0.43896489, -0.97277706, -0.46275165,  0.04488298],
        [ 0.46676669,  0.25570567, -0.483729  ,  0.31090798]],

       [[ 0.70017294,  0.36088056,  0.00660713, -0.41458494],
        [ 0.69846737,  0.1530454 , -0.05598784,  0.08796802],
        [ 0.72053302,  0.41312606,  0.05442396,  0.4552311 ],
        [ 0.37073076,  0.88553299, -0.26304428,  0.06061722],
        [ 0.08857632,  0.80257371,  0.12848218,  0.6681505 ],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.58678744,  0.88511824,  0.24966377, -0.03195321],
        [ 0.57823525,  0.91190705,  0.48963205,  0.70319907],
        [ 0.90271144,  0.63783082,  0.20981181,  0.05905335],
        [ 0.34117023,  0.03268828, -0.7082319 ,  0.01045338]],

       [[ 0.23034952,  0.34232126, -0.78572811, -0.59472727],
        [-0.51535784,  0.04729449, -0.61019772,  0.31626037],
        [-0.85273747, -0.14713671, -0.80994437, -0.78194519],
        [-0.51404307, -0.11827634, -0.47181769, -0.17484552],
        [ 0.12222278, -0.80596729, -0.62863628, -0.72242231],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [-0.04150979,  0.02593115,  0.02281716, -0.03806059],
        [ 0.11172013, -0.52007881, -0.62765913,  0.19653631],
        [-0.55208129,  0.10868437, -0.40834293,  0.01583014],
        [ 0.04600251, -0.33374088, -0.45430329, -0.40858156]],

       [[-0.18898351, -0.35900333,  0.41186396,  0.84380648],
        [-0.20973501,  0.16782677,  0.16954699, -0.24252793],
        [-0.8044472 ,  0.29944224, -0.29966399,  0.08453703],
        [ 0.04753721, -0.44019421, -0.24230842, -0.2439442 ],
        [-0.82328511,  0.102763  , -0.26894178,  0.61517129],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [-0.50780287, -0.17576997,  0.13840625, -0.22462695],
        [-0.4169882 , -0.40326314, -0.33913712,  0.22633081],
        [-0.08040422, -0.25614027, -0.60176736, -0.2958248 ],
        [-0.04911439,  0.51237386,  0.30555641,  0.14278864]],

       [[-0.29228618,  0.07032873, -0.42275461,  0.14351935],
        [ 0.31017237, -0.66825866, -0.35151221, -0.17749876],
        [ 0.68029822, -0.68855319, -0.12432248,  0.85059859],
        [ 0.76097184, -0.34337266,  0.23836962,  0.0216807 ],
        [-0.2031688 ,  0.48486107, -0.50210486,  0.42320441],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.18606039, -0.27499001, -0.13781183, -0.38948153],
        [ 0.35856569, -0.48940006,  0.05522391,  0.21885001],
        [ 0.83148673, -0.61264751,  0.40051714, -0.38604973],
        [ 0.79097371,  0.58072262, -0.25806073,  0.65131283]],

       [[ 0.6563184 ,  0.02992653,  0.62036982,  0.36051133],
        [ 0.8956753 ,  0.13246822, -0.50382819,  0.43335752],
        [-0.269877  ,  0.12963386,  0.08055496, -0.60937624],
        [ 0.1571839 ,  0.30663153,  0.32812596, -0.48062709],
        [ 0.72062723, -0.35322123, -0.15139047,  0.49093589],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.93713167, -0.05505147, -0.20027009, -0.63848003],
        [ 0.58483446, -0.35058626, -0.55669946, -0.74804236],
        [ 0.56710393, -0.70198986, -0.03531299,  0.6955412 ],
        [ 0.19480088,  0.46534131, -0.01288314,  0.42456557]],

       [[-0.68375026,  0.50306889, -0.2792584 ,  0.60378167],
        [ 0.08003578,  0.15964348, -0.22838222,  0.67146462],
        [ 0.27562948, -0.318043  ,  0.11891728, -0.07294208],
        [ 0.32058288, -0.26851826, -0.73932906,  0.34598769],
        [-0.17059999,  0.05279777, -0.21318583,  0.77415001],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.42705586,  0.27419659, -0.09743777,  0.0602282 ],
        [ 0.2547581 ,  0.08655888, -0.32290193,  0.26177876],
        [ 0.12934444,  0.39258715, -0.53617776, -0.12995405],
        [-0.7333593 ,  0.39677558, -0.66407637,  0.52366857]],

       [[ 0.25177431,  0.73549214,  0.94326994, -0.9071749 ],
        [ 0.7705504 ,  0.10834267,  0.21267147, -0.96243402],
        [ 0.26828007,  0.09774716, -0.23455436, -0.91854594],
        [ 0.83761039, -0.28904302,  0.81735591, -0.59127434],
        [ 0.28743217,  0.61764645,  0.58815474, -0.01798117],
        [ 0.        ,  0.        ,  0.        ,  0.        ],
        [ 0.24910934,  0.37356447,  0.52724866, -0.52776703],
        [ 0.6078405 , -0.02430418, -0.17231499, -0.71754265],
        [ 0.72219654,  0.45432029,  0.49247933,  0.02600384],
        [ 0.64491673,  0.664695  ,  0.10315818,  0.09240566]]])

Exercise 3#

Use sc.mean to compute the mean of the data for Alice along the Z dimension.
Do the same with numpy, what are the complications you encounter, that are not present when using the dataset?

Solution 3#

[18]:

help(sc.mean)

Help on function mean in module scipp.core.reduction:

mean(x: 'VariableLike', dim: 'Optional[str]' = None, *, out: 'Optional[VariableLike]' = None) -> 'VariableLike'
    Element-wise mean over the specified dimension.

    If the input has variances, the variances stored in the output are based on
    the "standard deviation of the mean", i.e.,
    :math:`\sigma_{mean} = \sigma / \sqrt{N}`.
    :math:`N` is the length of the input dimension.
    :math:`\sigma` is estimated as the average of the standard deviations of
    the input elements along that dimension.

    See :py:func:`scipp.sum` on how rounding errors for float32 inputs are handled.

    :param x: Input data.
    :param dim: Dimension along which to calculate the mean. If not
                given, the mean over all dimensions is calculated.
    :param out: Optional output buffer.
    :raises: If the dimension does not exist, or the dtype cannot be summed,
             e.g., if it is a string.
    :return: The mean of the input values.
    :seealso: :py:func:`scipp.nanmean`.

[19]:

mean = sc.mean(d['alice'], 'z')

When using numpy to compute the mean: - We must remember (or lookup) which dimension corresponds to the Z dimensions. - We need a separate call for values and variances. - We need to manually scale the variance with the inverse of the number of data points to get the variance of the mean (standard deviation of the mean scales with 1/sqrt(N)).

[20]:

np_values = np.mean(d['alice'].values, axis=0)
np_variances = np.mean(d['alice'].variances, axis=0)
np_variances /= d['alice'].shape[0]

[21]:

print(mean.values)
print(mean.variances)
print(np_values)
print(np_variances)

[[ 0.11751757  0.11704017  0.00172861  0.03438361]
 [ 0.27277308 -0.05816514 -0.26621712  0.07422365]
 [-0.02414047 -0.14206281 -0.19605875 -0.06645678]
 [ 0.2787862  -0.11777733 -0.01687275 -0.10859948]
 [-0.00064029  0.01466665 -0.18415795  0.29805201]
 [ 0.          0.          0.          0.        ]
 [ 0.24678863  0.11722692  0.12183395 -0.20460955]
 [ 0.24914471 -0.09290302 -0.2188483   0.04260094]
 [ 0.26017409 -0.11876651 -0.11769305  0.00368537]
 [ 0.21276963  0.32182018 -0.27157123  0.21844014]]
[[0.05008611 0.04447891 0.04413598 0.05023208]
 [0.04774672 0.04525981 0.04337504 0.05153547]
 [0.04621843 0.04096851 0.0477296  0.04855578]
 [0.04734272 0.0445698  0.04665032 0.05123232]
 [0.04766662 0.04461242 0.04443533 0.04841499]
 [0.04778026 0.04478924 0.04519545 0.05005479]
 [0.05093683 0.04634987 0.04519113 0.05180477]
 [0.04852958 0.04486588 0.04483412 0.04994213]
 [0.04963452 0.04479878 0.04589785 0.05476002]
 [0.05340728 0.0466263  0.04250211 0.04873229]]
[[ 0.11751757  0.11704017  0.00172861  0.03438361]
 [ 0.27277308 -0.05816514 -0.26621712  0.07422365]
 [-0.02414047 -0.14206281 -0.19605875 -0.06645678]
 [ 0.2787862  -0.11777733 -0.01687275 -0.10859948]
 [-0.00064029  0.01466665 -0.18415795  0.29805201]
 [ 0.          0.          0.          0.        ]
 [ 0.24678863  0.11722692  0.12183395 -0.20460955]
 [ 0.24914471 -0.09290302 -0.2188483   0.04260094]
 [ 0.26017409 -0.11876651 -0.11769305  0.00368537]
 [ 0.21276963  0.32182018 -0.27157123  0.21844014]]
[[0.05008611 0.04447891 0.04413598 0.05023208]
 [0.04774672 0.04525981 0.04337504 0.05153547]
 [0.04621843 0.04096851 0.0477296  0.04855578]
 [0.04734272 0.0445698  0.04665032 0.05123232]
 [0.04766662 0.04461242 0.04443533 0.04841499]
 [0.04778026 0.04478924 0.04519545 0.05005479]
 [0.05093683 0.04634987 0.04519113 0.05180477]
 [0.04852958 0.04486588 0.04483412 0.04994213]
 [0.04963452 0.04479878 0.04589785 0.05476002]
 [0.05340728 0.0466263  0.04250211 0.04873229]]