From tabular data to binned data#

[1]:

import plopp as pp
import scipp as sc

%matplotlib widget

Overview#

Binned data in Scipp is conceptually equivalent to an array of tables. In other words, it represents an array of records (table rows) sorted into an (often multi-dimensional) array of “bins”. In this tutorial we begin by learning how to setup tabular data appropriate for histogramming and binning with scipp. The main focus will then be binning the tabular data and basic usage of the resulting binned data.

We will use a file of a simulated neutron-scattering experiment — at the powder diffractometer DREAM at the European Spallation Source. The approach and techniques displayed here are however applicable far more generally and not specific to this scientific area.

Loading tabular data#

We will use a file created by a simulation for a diamond sample using McStas and Geant4. We can use scipp.io.load_csv to load the file as a dataset. The column headers of the file encode physical units in the form of tof [ns]. The 'bracket' header parser understands this format and extracts the units.

[2]:

filename = 'https://public.esss.dk/groups/scipp/scipp/1/data_dream_diamond.zip'
ds = sc.io.load_csv(
    filename,
    sep='\t',
    header_parser='bracket',
)
ds

This 1-D dataset represents the tabular data that was read from the file. In the above table, each row (record) describes an event, in this case the detection of a neutron, with its associated metadata such as the detector module or the x, y, and z position.

To histogram or bin data by a column, Scipp must know which columns are metadata and which column holds data values. The table is actually a table of metadata values for events with an implicit data value of “1 count” each. To continue we convert this into a data array:

[3]:

table = sc.DataArray(
    sc.ones(sizes=ds.sizes, unit='counts'),
    coords={name: column.data for name, column in ds.items()},
)
sc.table(table[:10])

[3]:

Coordinates													Data
counter	module	segment	strip	tof [ns]	voxel_x [mm]	voxel_y [mm]	voxel_z [mm]	wavelength [Å]	wire	x_pos [mm]	y_pos [mm]	z_pos [mm]	[counts]
2	12	6	20	73035529	963.085	-920.410	-1049.580	3.694	22	956.990	-922.167	-1045.120	1.000
1	4	5	20	72964368	1103.670	248.977	-889.281	3.702	4	1106.620	255.398	-892.764	1.000
2	14	5	20	73056111	704.337	-1196.640	-1094.100	3.691	27	703.154	-1204.100	-1102.690	1.000
2	14	5	19	73056111	704.337	-1196.640	-1107.980	3.691	27	702.934	-1202.910	-1104.840	1.000
1	10	1	19	73024979	1075.960	-768.968	-1053.870	3.694	21	1080.410	-766.272	-1051.010	1.000
2	7	5	19	72964749	1160.890	-179.184	-936.635	3.700	8	1161.530	-173.722	-935.843	1.000
1	13	3	19	72964010	634.259	-963.880	-918.598	3.701	6	628.887	-966.418	-911.440	1.000
1	13	3	20	72964010	634.259	-963.880	-907.091	3.701	6	629.122	-965.821	-910.524	1.000
1	1	4	15	73099429	972.847	661.239	-984.737	3.705	8	970.896	665.640	-984.516	1.000
1	7	2	19	72961966	1134.810	-263.814	-927.616	3.700	7	1133.400	-269.292	-929.414	1.000

Binning with edges and groups combined#

It is also possible to combine binning and grouping. Since strips roughly correspond to scattering angle, a plot against wavelength and strip may be useful.

Exercise 5#

Use group table by ‘strip’ and bin by ‘wavelength’.
Plot the result.

Solution#

[23]:

binned_strip_wavelength = table.group('strip').bin(wavelength=1000)
binned_strip_wavelength.hist().plot(norm='log')

[23]:

From tabular data to binned data#

Overview#

Loading tabular data#

Histogramming and binning#

Exercise 1#

Solution#

Multi-dimensional spatial binning#

Exercise 2#

Solution#

Multi-dimensional logical binning#

Exercise 3#

Solution#

From event-based metadata to bin-based metadata#

Exercise 4#

Solution#

Binning with edges and groups combined#

Exercise 5#

Solution#