Understanding Event Data#

Introduction#

Neutron-scattering data may be recorded in “event mode”: For each detected neutron a (pulse) timestamp and a time-of-flight is stored. This notebook will develop an understanding of how do work with this type of data.

Our objective is not to demonstrate or develop a full reduction workflow. Instead we develop understanding of data structures and opportunities that event data provides.

This tutorial contains exercises, but solutions are included directly. We encourage you to download this notebook and run through it step by step before looking at the solutions. Event data is a particularly challenging concept so make sure to understand every aspect before moving on. We recommend to use a recent version of JupyterLab: The solutions are included as hidden cells and shown only on demand.

We use data containing event data from the POWGEN powder diffractometer at SNS. Note that the data has been modified for the purpose of this tutorial and is not entirely in its original state. We begin by loading the file and plot the raw data:

[1]:

import scipp as sc
import scippneutron as scn
import plopp as pp

dg = scn.data.tutorial_event_data()
dg

Downloading file 'powder-event-filtered.h5' from 'https://public.esss.dk/groups/scipp/scippneutron/5/powder-event-filtered.h5' to '/home/runner/.cache/scippneutron/5'.

[1]:

scipp.DataGroup

(spectrum: 10694, tof: 1, time: 330473)

events
scipp
DataArray
(spectrum: 10694, tof: 1)
DataArrayView
binned data [len=704, len=596, ..., len=0, len=0]
proton_charge
scipp
DataArray
(time: 330473)
float64
pC
1.484e+07, 1.484e+07, ..., 1.487e+07, 1.484e+07

[2]:

events = dg['events']
events.hist(spectrum=500, tof=400).plot()

[2]:

../_images/tutorials_3_understanding-event-data_3_0.svg

We can see some diffraction lines, but they are oddly blurry. There is also an artifact from the prompt-pulse visible at \(4000~\mu s\). This tutorial illustrates how event data gives us the power to understand and deal with the underlying issues. Before we start the investigation we cover some basics of working with event data.

From binned data to dense data#

While we often want to perform many operations on our data in event mode, a basic but important step is transformation of event data into dense data, since typically only the latter is suitable for data analysis software or plotting purposes. There are two options we can use for this transformation, described in the following.

Option 2: Histogramming#

For performance and memory reasons binned data often contains the minimum number of bins that is “necessary” for a given purpose. In this case da only contains a single time-of-flight bin (essentially just as information what the lower and upper bounds are in which we can expect events), which is not practical for downstream applications such as data analysis or plotting.

Instead of simply summing over all events in a bin we may thus histogram data. Note that scipp makes the distinction between binning data (preserving all events individually) and histogramming data (summing all events that fall inside a bin).

For simplicity we consider only a single spectrum:

[10]:

spec = events['spectrum', 8050]
sc.show(spec)

[11]:

sc.table(spec.values[0]['event', :5])

[11]:

Coordinates		Data
pulse_time [ns]	tof [µs]	[counts]
2011-08-12T15:50:46.309402465	13840.203	1.000±1.000
2011-08-12T15:50:47.959114074	2999.387	1.000±1.000
2011-08-12T15:50:53.675106048	14104.980	1.000±1.000
2011-08-12T15:51:34.688072204	2829.583	1.000±1.000
2011-08-12T15:52:05.841064453	2887.581	1.000±1.000

Note the chained slicing above: We access the zeroth event list and select the first 5 slices along the event dimension (which is the only dimension, since the event list is a 1-D table).

We use one of the scipp functions for creating a new variable to define the desired bin edge of our histogram. In this case we use sc.linspace (another useful option is sc.geomspace):

[12]:

tof_edges = sc.linspace(dim='tof', start=18.0, stop=17000, num=101, unit='us')
spec.hist(tof=tof_edges).plot()

[12]:

../_images/tutorials_3_understanding-event-data_24_0.svg

Exercise#

Change tof_edges to control what is plotted:

Change the number of bins, e.g., to a finer resolution.
Change the start and stop of the edges to plot only a smaller time-of-flight region.

Solution#

[13]:

tof_edges = sc.linspace(dim='tof', start=2000.0, stop=15000, num=201, unit='us')
spec.hist(tof=tof_edges).plot()

[13]:

../_images/tutorials_3_understanding-event-data_26_0.svg

Higher dimensions and cuts#

For purposes of plotting, fitting, or data analysis in general we will typically need to convert binned data to dense data. We discussed the basic options for this in From binned data to dense data. In particular when dealing with higher-dimensional data these options may not be sufficient. For example we may want to:

Create a 1-D or 2-D cut through a 3-D volume.
Create a 2-D cut but integrate over an interval in the remaining dimension.
Create multi-dimensional cuts that are not aligned with existing binning.

All of the above can be achieved using tools we have already used, but not all of them are covered in this tutorial.

Exercise#

Adapt the above code used for binning and masking the single spectrum (spec) along pulse_time and tof to the full data array (da).

Hint: This is trivial.

Solution#

[28]:

%matplotlib widget
binned_da = events.bin(tof=prompt_tof_edges, pulse_time=pulse_time_edges)
binned_da.masks['prompt_pulse'] = prompt_mask
binned_da.masks['bad_beam'] = pulse_time_mask
pp.slicer(
    binned_da.transpose(['pulse_time', 'spectrum', 'tof']).hist(spectrum=500, tof=400)
)

[28]:

Making a 2-D cut#

Exercise#

Adapt the code of the 1-D cut to create 100 two_theta bins.
Make a 2-D plot (with dspacing and two_theta on the axes).

Solution#

[37]:

two_theta_cut = sc.linspace(dim='two_theta', unit='rad', start=0.4, stop=1.0, num=101)
dspacing_cut = sc.linspace(dim='dspacing', unit='Angstrom', start=0.0, stop=2.0, num=2)
pulse_time_cut = tmin + sc.array(dims=['pulse_time'], unit='s', values=[0, 10 * 60]).to(
    unit='ns'
)
cut = da_dspacing.bin(
    two_theta=two_theta_cut, dspacing=dspacing_cut, pulse_time=pulse_time_cut
)
cut = cut['pulse_time', 0]  # squeeze pulse time (dim of length 1)
dspacing_edges = sc.linspace(
    dim='dspacing', unit='Angstrom', start=0.0, stop=2.0, num=1000
)
cut = cut.hist(dspacing=dspacing_edges)
cut.plot()

[37]:

Understanding Event Data#

Introduction#

Inspecting event data#

Exercise#

From binned data to dense data#

Option 1: Summing bins#

Option 2: Histogramming#

Exercise#

Solution#

Masking event data — Binning by existing parameters#

Masking events#

Exercise#

Solution#

Masking bins#

Bonus question#

Solution#

Binning by new parameters#

Exercise#

Solution#

Higher dimensions and cuts#

Exercise#

Solution#

Removing binned dimensions#

Making a 1-D cut#

Exercise#

Making a 2-D cut#

Exercise#

Solution#