# Concepts#

## DataArray and Dataset meta data handling#

This section describes details about how coords (and masks) of datasets and data arrays behave when slicing, combining, or inserting.

[1]:

import numpy as np
import scipp as sc

x = sc.Variable(dims=['x'], values=[1,2,3,4])
da = sc.DataArray(data=x,
coords={'x':x},
ds = sc.Dataset(data={'a':da})


Consider a data array da and a dataset ds with an aligned coord and an aligned mask. The following conditions must hold:

[2]:

assert da['x', 0:1].coords['x'].aligned  # range slice preserves coord and alignment
assert not da['x', 0].coords['x'].aligned  # point slice makes coord unaligned

[3]:

assert sc.identical(ds['a']['x', 0:1], ds['x', 0:1]['a'])
assert sc.identical(ds['a']['x', 0], ds['x', 0]['a'])

[4]:

assert ds['a'].coords['x'].aligned
assert ds['x', 0:1].coords['x'].aligned
assert not ds['x', 0].coords['x'].aligned



In operations, aligned coords are compared:

[5]:

try:
ok = da['x', 0:1] + da['x', 1:2]
except:
ok = False
assert not ok


Mismatching attrs and unaligned coords are dropped:

[6]:

assert sc.identical(da + da['x', 1], da + da['x', 1].data)


[7]:

assert not sc.identical(da + da['x', 0], da + da['x', 0].data)


A missing attr is interpreted as mismatch to ensure that:

[8]:

a = da['x', 0].copy()
a.attrs['x'] = a.coords.pop('x')
b = da['x', 1].copy()
b.attrs['x'] = b.coords.pop('x')
c = da['x', 2].copy()
c.attrs['x'] = c.coords.pop('x')
assert sc.identical(a + (b + c), (a + b) + c)


Likewise, a missing unaligned coord is interpreted as mismatch:

[9]:

a = da['x', 0].copy()
b = da['x', 1].copy()
c = da['x', 2].copy()
assert sc.identical(a + (b + c), (a + b) + c)


Aligned coords take precedence over unaligned coords:

[10]:

a = da['x', 0].copy()
a.coords.set_aligned('x', True)
b = da['x', 1].copy()
assert sc.identical((a + b).coords['x'], a.coords['x'])


Insertion order does not matter for attrs:

[11]:

a = da.copy()
a.attrs['attr'] = 1.0 * sc.units.m
b = da.copy()
b.attrs['attr'] = 2.0 * sc.units.m
ds1 = sc.Dataset()
ds2 = sc.Dataset()
ds1['a'] = a
ds1['b'] = b
ds2['b'] = b
ds2['a'] = a
assert sc.identical(ds1, ds2)


Insert into dataset with mismatching attrs drops attr:

[12]:

ds = sc.Dataset()
ds.coords['x'] = x['x', 0]
ds['a'] = da['x', 1] # Drops 'x' from 'a'
assert sc.identical(ds.coords['x'], ds['a'].coords['x']) # shadowing is NOT supported


Masks of dataset items are independent:

[13]:

ds = sc.Dataset()


If there is no coord, the attr is preserved for all items. Adding a coord later makes the meta property invalid because of ambiguous name shadowing:

[14]:

ds = sc.Dataset()
a = da['x', 0].copy()
a.attrs['x'] = a.coords.pop('x')
ds['a'] = a
b = da['x', 1].copy()
b.attrs['x'] = b.coords.pop('x')
ds['b'] = b
assert 'x' not in ds.coords
assert 'x' in ds['a'].attrs
assert 'x' in ds['b'].attrs
ds.coords['x'] = x['x', 0] # introduce shadowing
try:
ds['a'].meta # raises because of shadowing
except:
ok = True
else:
ok = False
assert ok
del ds.coords['x']

[15]:

edges = sc.Variable(dims=['x'], values=[1,2,3,4,5])
da.coords['x'] = edges
assert sc.identical(sc.concat([da['x', :2], da['x', 2:]], 'x'), da)
assert sc.identical(sc.concat([da['x', 0], da['x', 1]], 'x'), da['x', 0:2])
assert sc.identical(sc.concat([da['x', :-1], da['x', -1]], 'x'), da)
da_yx = sc.concat([da['x', :2], da['x', 2:]], 'y') # create 2-D coord
assert sc.identical(da_yx.coords['x'], sc.concat([da.coords['x']['x', :3], da.coords['x']['x', 2:]], 'y'))


2-D coords for a dimension prevent operations between slices that are not along that dimension:

[16]:

da_2d = sc.DataArray(
data=sc.zeros(dims=['y', 'x'], shape=[2, 2]),
coords={
'x':sc.Variable(dims=['y', 'x'], values=np.array([[1, 2], [3, 4]])),
'y':sc.Variable(dims=['y'], values=[3, 4])})

da_2d['x', 0] + da_2d['x', 1] # Same as with 1-D coord: x-coord differs but not aligned due to slice.
try:
# 'y' sliced, so 'x' coord is aligned and yields different values from slices of 2-D coord.
da_2d['y', 0] + da_2d['y', 1]
except RuntimeError:
ok = False
else:
ok = True
assert not ok


Coords cannot be added or erased via items since a new coord dict is created when getting a dataset item:

[17]:

try:
ds['a'].coords['fail'] = 1.0 * sc.units.m
except sc.DataArrayError:
ok = False
else:
ok = True
assert not ok
assert 'fail' not in ds.coords

[18]:

ds.coords['xx'] = 1.0 * sc.units.m
assert 'xx' in ds['a'].coords
try:
del ds['a'].coords['xx']
except sc.DataArrayError:
ok = False
else:
ok = True
assert not ok
assert 'xx' in ds.coords


The same mechanism applies for coords, masks, and attrs of slices:

[19]:

try:
da['x', 0].coords['fail'] = 1.0 * sc.units.m
except sc.DataArrayError:
ok = False
else:
ok = True
assert not ok
assert 'fail' not in da.coords


meta contains dataset coordinates as well as item attributes, cannot add or erase, since ambiguous:

[20]:

try:
ds['a'].meta['fail'] = 1.0 * sc.units.m
except sc.DataArrayError:
ok = False
else:
ok = True
assert not ok
assert 'fail' not in ds['a'].meta

[21]:

ds['a'].attrs['attr'] = 1.0 * sc.units.m
assert 'attr' in ds['a'].meta
try:
del ds['a'].meta['attr']
except sc.DataArrayError:
ok = False
else:
ok = True
assert not ok
assert 'attr' in ds['a'].meta


Attributes are independent for each item, and show up in meta of the items:

[22]:

ds['a'].attrs['attr'] = 1.0 * sc.units.m
ds['b'].attrs['attr'] = 2.0 * sc.units.m
assert 'attr' in ds['a'].meta
assert 'attr' in ds['b'].meta
assert 'attr' not in ds.meta
assert not sc.identical(ds['a'].attrs['attr'], ds['b'].attrs['attr'])
del ds['a'].attrs['attr']
del ds['b'].attrs['attr']