Release Notes
Contents
Release Notes#
v0.13.1 (April 2022)#
Bugfixes#
Fix exception in
scipp.islinspace()
for dtypes other than float64 #2523.
v0.13.0 (March 2022)#
Features#
Added
is_edges
method tocoords
,attrs
,masks
, andmeta
properties of DataArray and Dataset #2469.Added support for arithmetic of
scipp.DataArray
andscipp.Dataset
with builtinint
andfloat
#2473.Added advanced indexing support: integer array indexing #2502.
Added advanced indexing support: boolean variable indexing #2477.
Added or exposed to Python more functions for boolean logic #2480, #2489.
broadcast
now supportsscipp.DataArray
, and thesizes
argument #2488.The output of
squeeze()
is no longer read-only 2494.Added support for spatial dtypes to
scipp.to_unit()
andscipp.isclose()
#2498.
Breaking changes#
Changed format of scipp native HDF5 files to allow for UTF-8 characters in coord names; migration script:
tools/migration/scipp-0.13-hdf5-files.py
#2463.
Bugfixes#
Fix permissions error for configuration file where users do not have write access to their config dir #2482.
Fix plotting of data with multi-dimension bin-edge coordinate in the presence of a mask that does not depend on the bin-edge dimension #2505.
Fix incorrect axis labels in plots with datetime tick labels #2507.
Fix segfault when calling
scipp.GroupByDataArray.copy()
orscipp.GroupByDataset.copy()
with out of bounds index #2511.
Contributors#
Simon Heybrock a, Neil Vaytet a, and Jan-Lukas Wynen a
v0.12.1 (February 2022)#
Bugfixes#
Fix a plotting bug when 2d data has masks and a coord with no unit #2470.
Contributors#
Simon Heybrock a, Neil Vaytet a, and Jan-Lukas Wynen a
v0.12.0 (February 2022)#
Features#
Added support for
abs()
toscipp.DataArray
,scipp.Dataset
,scipp.Variable
#2382.scipp.bin()
will now also look at attributes to perform the binning, instead of only looking at the coordinates #2388.Added
scipp.squeeze()
to remove length-1 dimensions #2385.Switched to new system for runtime configuration, see the corresponding section in the reference documentation #2389.
Added
scipp.midpoints()
which can be used to compute bin-centers #2404.scipp.fold()
now supports-1
for one of the dimension sizes, to indicate automatic shape #2414.Added
quiet
argument toscipp.transform_coords()
#2420.Added support for strides for positional slicing. Negative strides are not supported for now #2423.
scipp.zeros_like()
,scipp.ones_like()
,scipp.empty_like()
, andscipp.full_like()
now accept and return data arrays #2425.Added the
.rename
method toscipp.Variable
,scipp.DataArray
, andscipp.Dataset
that renames both dimensions and coords/attrs at the same time #2428.
Breaking changes#
The configuration option
config['plot']['padding']
is now calledconfig['plot']['bounding_box']
, no action is required if it was never modified #2389.scipp.rebin()
now applies masks that depend on the rebinning dimension to avoid “growing” effect on masks which previously resulted in masking more than intended #2383.The
unit
attribute now distinguishesNone
fromdimensionless
#2395 and #2417.For number-like dtypes the default is
dimensionless
(unchanged) whereas all other dtypes such as strings default toNone
(new).bool
(used for masks) is not considered number-like and the default unit isNone
.Comparison operator results now use
None
as unit of their output.When integers without unit are used as indices, the new
scipp.index()
can be used to create a scalar variable without unit.
scipp.spatial.rotations_from_rotvecs()
has been changed to taking a variable input rather than separate values/dims/units.scipp.NotFoundError
error has been replaced withKeyError
in Python #2416.Some functions now raise
scipp.DimensionError
instead ofscipp.NotFoundError
#2416.
Bugfixes#
Fix
scipp.full()
, which previously converted or attempted to convert all values tofloat64
unless thedtype
was specified explicitly #2395.Fix
scipp.transform_coords()
for sliced binned data, used to raise an exception #2406.Fix 1d plots y-axis limits when toggling log-scale with buttons and fix 2d plot colorbar when Inf values #2426.
Fix bug leading to datasets dims/shape modification after failed item insertion #2437.
Fix a plotting bug when 2d data has y coord associated with dimension x #2446.
Fix plotting with custom tick labels of
dtype=datetime64
which previously raised an exception #2449.
Contributors#
Simon Heybrock a, Neil Vaytet a, Tom Willemsen b, c, and Jan-Lukas Wynen a
v0.11.0 (January 2022)#
Features#
Added new datatypes for representing spatial transformations:
affine_transform3
,rotation3
, andtranslation3
. Each of these can be combined and applied to the existing vector datatype.Added support for slicing without specifying a dimension (only for 1-D objects) #2321.
Added
scipp.interpolate.interp1d()
as convenience wrapper of the scipy functions of the same name #2324.Added
scipp.integrate.trapezoid()
andscipp.integrate.simpson()
as convenience wrappers of the scipy functions of the same name #2324.Added
unit
property toobj.bins
for getting and setting unit of bin elements #2330.Added
scipp.optimize.curve_fit()
as convenience wrappers of the scipy function of the same name #2350.Added
scipp.signal.butter()
andscipp.signal.sosfiltfilt()
as wrappers of the scipy functions of the same name #2356.Added
scipp.Variable.to()
andscipp.DataArray.to()
as a convenience function for simultaneously converting dtype and units.Added
scipp.datetime()
,scipp.datetimes()
, andscipp.epoch()
to conveniently construct variables containing datetimes #2360.Added
camera
option for 3-D scatter plots to control camera position and the point the camera is looking at #2361.Allow more interoperability between
scipp.DType
and other ways of encoding dtypes #2373.
Breaking changes#
Minor change in behavior of dimension renaming in
transform_coords
, new behavior documented in Coordinate transformations. Most use cases are unaffected. #2319.sc.spatial.transform.as_rotvec
has been moved toscipp.spatial.rotation_as_rotvec()
sc.spatial.transform.from_rotvec
has been moved toscipp.spatial.rotations_from_rotvecs()
, and now returns a rotation data type rather than a matrix. For consistency with other transformation creation functions,rotations_from_rotvecs
now takesvalues
,dims
, andunit
separately.The matrix dtype
matrix_3_float64
has been renamed tolinear_transform3
, and should now be constructed withscipp.spatial.linear_transform()
.The vector dtype
vector_3_float64
has been renamed tovector3
.Scipp’s logger is no longer preconfigured, this has to be done by the user #2372.
Data types are now attributes of the
scipp.DType
class and thescipp.dtype
module has been removed #2373.
Bugfixes#
Fix coordinate and attribute comparisons to treat NaN (not-a-number) values as equal, which previously prevented most operations with data arrays or datasets that contained NaN values in their coordinates or attributes #2331.
Fix incorrect result from
sc.bin
when rebinning an inner dimension to a multi-dimensional coordinate #2355.
Deprecations#
sc.matrix
andsc.matrices
have been deprecated in favour ofscipp.spatial.linear_transform()
andscipp.spatial.linear_transforms()
, and will be removed in a future release of scipp.The previously deprecated
concatenate
andgroupby(..).concatenate
have been removed. Usescipp.concat()
andscipp.GroupByDataArray.concat()
instead.
Contributors#
Simon Heybrock a, Neil Vaytet a, Tom Willemsen b, c, and Jan-Lukas Wynen a
v0.10.0 (December 2021)#
Features#
Bugfixes#
Fix kernel crash or poor performance when using
bin
orda.bins.concat
along an inner dimension in the presence of a long outer dimension #2278.Improved the rendering of cut slices in 3d plots where some strange artifacts could appear depending on the order in which the pixels were being drawn #2287.
Deprecations#
The deprecated
events
property has been removed.
Stability, Maintainability, and Testing#
Contributors#
Simon Heybrock a, Neil Vaytet a, Tom Willemsen b, c, and Jan-Lukas Wynen a
v0.9.0 (November 2021)#
Features#
Added
erf
anderfc
functions #2195.Reduction operations such as
sum
,nansum
, andcumsum
of single precision (float32) data now use double precision (float64) internally to reduce rounding errors #2218.Added
bins_like
for broadcasting dense variables to binned variables, e.g., for converting bin coordinates into event coordinates #2225.groupby
now also supports grouping by attributes instead of just by coordinates #2227.Added
concat
to replaceconcatenate
. In contrast to the now deprecatedconcatenate
,concat
supports concatenation of lists of objects instead of just two objects #2232.Added
dim
property toVariable
andDataArray
#2251.Added preconfigured logging support including a widget for output to notebooks #2255, #2267.
Added
reduce
for setting up reductions across a list of inputs #2267.Added toggle buttons for hiding/showing the 3d axes and outline box in 3d plots #2265.
Breaking changes#
Bugfixes#
Fix bugs in
rebin
if data and/or edges had strides other than 1 along rebinned dimension, typically only occurring with multi-dimensional (ragged) coordinates #2211.Fix exception that was thrown when importing empty datasets from HDF5 files using
open_hdf5
#2216.Fix exception in
astype
when called with binned data that does not require conversion #2222.Fix bug in
concatenate
that could lead to masks being shared with input rather than being copied #2232.Fix exception in
bin
when binning in a new dimension but with an existing bin coord #2237.Fix exception when plotting data with masks in presence of multi-dimensional coords #2269.
Deprecations#
concatenate
andgroupby(..).concatenate
are deprecated. Useconcat
andgroupby(..).concat
instead.
Stability, Maintainability, and Testing#
More thorough unit testing for low-level code underlying most operations on dense as well as binned data #1900, by Jan-Lukas Wynen.
Switch to Jupyter Book theme for documentation, avoiding a number of issues with the RTD theme #2233, by Neil Vaytet.
Added benchmark suite to allow for identification of performance regressions, by Samuel Jones.
Added code coverage tooling to allow for identification of dark spots in our test coverage, by Simon Heybrock.
Contributors#
Owen Arnold b, c, Simon Heybrock a, Samuel Jones b, Neil Vaytet a, Tom Willemsen b, c, and Jan-Lukas Wynen a
v0.8.3 (September 2021)#
Bugfixes#
Fix serious correctness bug in
groupby.sum
andgroupby.mean
#2200.
v0.8.0 (September 2021)#
Features#
Added
sizes
argument tozeros
,ones
, andempty
variable creation functions #1951.Slicing syntax now supports ellipsis, e.g.,
da.data[...] = var
#1961.Added bound method equivalents to many free functions which take a single Variable or DataArray #1969.
Variables can now be constructed directly from multi dimensional lists and tuples #1977.
Plotting 1-D event data is now supported #2018.
Add
transform_coords
for (multi-step) transformations based on existing coords, with support of event coords #2058.Add
from_pandas
andfrom_xarray
for conversion of pandas dataframes, xarray data arrays and dataset to scipp objects #2054.Added
full
andfull_like
variable creation functions #2069.islinspace
can now take multi-dimensional variables as long as you pass the dimension to be checked #2094.Added a power function and support for the
**
operator #2083.Binned data now has a
mean
method as well assum
, which returns the mean of each element within a bin.Add
scipp.constants
module for physical constants #2101.Add
scipp.spatial.transform
module providingfrom_rotvec
andas_rotvec
#2102.Add
cross
function to complementing existingdot
function #2109.The
fields
property of structured variables (vector and matrix dtypes) can now be iterated and provides dict-like access #2116.Add
where
function.Unary operations such as
sin
are now available for datasets as well #2112.Add rounding functions
floor
,ceil
, andround
that performs similarly to numpy equivelants #2147.
Breaking changes#
Changed names of arithmetic functions to match numpy’s names:
plus
→scipp.add()
,minus
→scipp.subtract()
,times
→scipp.multiply()
#1999.Changed Variable init method, all arguments are keyword-only, special overloads which default-initialize data were removed #1994.
The
axes
keyword arg ofplot
has been removed. Uselabels
to define labels andtranspose
to transpose instead #2018.The
redraw
method of plots does not support replacing data variables of data arrays any more, but only updates of data values #2018.Made most arguments of
DataArray
andDataset
constructors keyword-only #2008.Made arguments of many functions keyword-only, e.g. constructors of
DataArray
andDataset
,bin
, andatan2
#1983, #2008, #2012.astype
andto_unit
now copy the input by default even when no transformation is required, use the newcopy
argument to avoid #2016.rename_dims
does not rename dimension-coords any more, only dims will be renamed #2058.rename_dims
does not modify the input anymore but returns a new object with renamed dims #2058.issorted
andislinspace
return a variable of type boolean instead of a boolean #2094.Multi-dimensional coordinates are no longer associated with their inner dimension, which affects the behavior when slicing: Such coords will no longer be turned into attributes during a slice operation on the inner dimension #2098.
buckets.map(histogram, da.data, 'time')
has been replaced bylookup(histogram, 'time')[da.bins.coords['time']]
#2112.rebin
andhistogram
now support any unit.plot()
guesses which resampling mode to use and provides a button to toggle this #2180.
Bugfixes#
Deprecations#
The
events
property is deprecated and is scheduled for removal with v0.9.
Contributors#
Owen Arnold b, c, Simon Heybrock a, Samuel Jones b, Greg Tucker a, Neil Vaytet a, Tom Willemsen b, c, and Jan-Lukas Wynen a
v0.7.2 (September 2021)#
Bugfixes#
Fix serious correctness bug in
groupby.sum
andgroupby.mean
#2200.
v0.7.0 (June 2021)#
Features#
Licence changed from GPLv3 to BSD-3-Clause.
Can now control the position and visibility of the legend in 1d plots with
plot(da, legend={"show": True, "loc": 4})
#1790.Added
zeros_like
,ones_like
andempty_like
functions #1864.sort
is now considerably faster for data with more rows #1872.Added numpy-like
linspace
,logspace
,geomspace
, andarange
variable creation functions #1871.to_unit
now supports binned data #1841.broadcast
now returns a view instead of copying #1861.fold
now returns a view instead of copying #1861.fold
now works for binned data #1861.flatten
now returns a view of the input buffer if the input data is contiguous #1861.Add
redraw
method to plot which enables an easy update on a figure when the underlying data has been modified #1907.Several improvements for work with (3-D position) vectors and (3-D rotation) matrices #1905:
Add creation functions
vector
,vectors
,matrix
,matrices
.Direct creation and initialization of 2-D (or higher) arrays of matrices and vectors is now possible from numpy arrays.
Fix extremely slow initialization of array of vectors or matrices from numpy arrays.
The
values
property now returns a numpy array withndim+1
(vectors) orndim+2`
(matrices) axes, with the inner 1 (vectors) or 2 (matrices) axes corresponding o the vector or matrix axes.Vector or matrix element can now be accessed and modified directly using the new
fields
property ofVariable
. Thefields
property provides propertiesx
,y
,z
(for variables containing vectors) orxx
,xy
, …,zz
(for matrices) that can be read as well as set.
Reduction operations such as
sum
andmean
are now also multi-threaded and thus considerably faster #1923.
Bugfixes#
Profile plots are no longer disabled for binned data with only a single bin in the 3rd dimension #1936.
Breaking changes#
Scipp’s data structures now behave mostly like normal nested Python objects, i.e., copies are shallow by default #1823.
filter
andsplit
removed. Identical functionality can be achieved usinggroupby
and/or slicing.reshape
has been removed. Usefold
andflatten
instead #1861.geometry.x
,geometry.y
, andgeometry.z
have been removed. Use theVariable.fields
property instead #1925.
Contributors#
Owen Arnold b, c, Simon Heybrock a, Matthew D. Jones b, c, Neil Vaytet a, and Jan-Lukas Wynen a
v0.6.1 (April 2021)#
Bugfixes#
map
andscale
operations as well ashistogram
for binned data now also work withdatetime64
#1834.bin
now works on previously binned data with 2-D edges, even if the outer dimensions(s) are not rebinned #1836.bin
andhistogram
now work withint32
-valued coordinates and support binning withint64
- orint32
-valued bin edges.
v0.6.0 (March 2021)#
Features#
Add
sizes
properties, an ordereddict
to lookup the size of a specific dimension #1636.Add access to data, coords, attrs, and masks of bins via the
bins
property, e.g.,obj.bins.coords['x']
#1752.Add more functions for data arrays that were previously available only for variables #1638 #1660.
Add named versions of operators such as
logical_and
#1660.Add
to_unit
to converter variables, e.g., betweenm
andmm
#1756.Add modulo operations #1660.
Add scaling operations for Variables of type
vector_3_float64
.sum
andmean
implemented for Variables of typevector_3_float64
.Add
fold
andflatten
which allow to reshape dimensions of a Variable or DataArray #1676.It is now possible to reshape a Variable also with
sc.reshape(var, sizes={'x': 2, 'y': 3})
, in addition tosc.reshape(var, dims=['x', 'y'], shape=(2, 3))
.Add
ones
andempty
creation functions, similar to what is known from numpy #1732.scipp.neutron
has been removed and is replaced by scippneutronscipp.neutron
(nowscippneutron
)isclose
added (is_approx
removed). Fuzzy data comparisonisclose
is an analogue to numpy’sisclose
stddevs
added #1762.Support for datetime
Breaking changes#
The
plot
module has been renamed toplotting
, and plotting is now achieved viasc.plot(data)
. Wrapper functionssuperplot
,image
, andscatter3d
have been dropped #1696.Properties
dim
,begin
,end
, anddata
of thebins
property of and object are now accessible as a dict viaobj.bins.constituents
.scipp.neutron
has been removed and is replaced by scippneutronscipp.neutron
(nowscippneutron
)is_equal
renamedidentical
is_linspace
renamedislinspace
is_sorted
renamedissorted
is_approx
superseded byisclose
.is_approx
removed.Removed support for facility-specific
convert_with_calibration
andload_calibration
Contributors#
Matthew Andrew, Owen Arnold, Simon Heybrock, Matthew D. Jones, Andrew McCluskey, Neil Vaytet, and Jan-Lukas Wynen
v0.5.0 (January 2021)#
Features#
New
profile
plotting functionality where one of the slider dimensions can be displayed as a profile in a subplot (1-D and 2-D projections only).Sliders have a thickness slider associated with them and can be used to show slices of arbitrary thickness.
Can hide/show individual masks on plots.
Can toggle log scale of axes and colorbar with buttons in figure toolbar.
Add binned data support, replacing “event list” dtypes as well as “realign” support.
Plotting of event data (binned data) with dynamic resampling with “infinite zoom” functionality.
Value-based slicing support.
Possibility to plot Scipp objects using
my_data_array.plot()
in addition to the classicalplot()
free function.Support for saving and loading scipp data structures to HDF5.
More functions such as
nanmean
for better handling of special values such asINF
andNaN
.TBB (multi-threading) support for MacOS.
scipp.neutron
Improved instrument view, e.g., with buttons to align camera with an axis.
Experiment logs (previously using Mantid’s
Run
) are now represented as native scipp objects, e.g., as scalar attributes holding a data array representing a time-series such as a temperature log.Support conversion of
mantid.MaskWorkspace
.
Breaking changes#
Dataset
does not have amasks
property any more. Useds['item'].masks
instead.Dataset
does not support attributes any more.DataArray
and dataset item attributes are now are now handled as “unaligned” coords. Useds['item'].coords
orarray.attrs
to access these.API for log scale on axes and colors has changed. Use
plot(da, scale={'tof': 'log'})
to set a log scale on a coordinate axis, and useplot(da, norm='log')
to have a log image colorscale or a log y axis on a 1d plot.vmin
andvmax
now represent absolute values instead of exponents whennorm='log'
.The
ipympl
matplotlib backend is now required for using inside Jupyter notebooks. This has been added as a dependency. It is also the only interactive backend that works in JupyterLab.Removed support for
event_list
dtype
, use binned data instead.Removed support for “realigned” data. This is replaced by the more flexible and generic support for “binned” data.
Contributors#
Matthew Andrew, Owen Arnold, Thibault Chatel, Simon Heybrock, Matthew D. Jones, Daniel Nixon, Piotr Rozyczko, Neil Vaytet, and Jan-Lukas Wynen
v0.4 (July 2020)#
Features#
New realign functionality.
Support for event-filtering.
Support for subtraction and addition for (realigned) event data.
Non-range slicing changed to preserve coords as attrs rather than dropping
scipp.neutron
: Instrument view with advanced geometry support, showing correct pixel shapes.Instrument view working on doc pages.
Made it simpler to add new
dtype
and supporttransform
for all types.Comparison functions such as
less
,greater_equal
, …all
andany
can work over all dimensions as well as explicitly provided dimension argumentIt is now possible to convert between Scipp objects and Python dictionaries using
to_dict
andfrom_dict
.New functions
collapse
andslices
can be use to split one or more dimensions of a DataArray to a dict of DataArrays.You can now inspect the global object list of via the
repr
for scipp showing Datasets, DataArrays and VariablesInternal cleanup and documentation additions.
Noteable bug fixes#
Several fixes in the plotting (non-regular bins, colorbar limits, axes tick labels from unaligned coordinates, etc…)
Breaking changes#
Coord and attributes names for neutron data have been standardized, now using hyphens instead of underscore, except for subscripts. Affected examples:
pulse-time
(previouslypulse_times
),source-position
(previouslysource_position
),sample-position
(previouslysample_position
),detector-info
(previouslydetector_info
).scipp.neutron.load
must useadvanced_geometry=True
option for loadingdetector-info
and pixel shapes.Normalization of event data cannot be done directly any more, must use
realign
.Plotting variances in 2D has been removed, and the API for using
matplotlib
axes has been simplified slightly, since we no longer have axes for variances:Before:
plot(..., mpl_axes={"ax": myax0, "cax": myax1})
After:
plot(..., ax=myax0, cax=myax1)
Plot with keyword argument
collapse
has been removed in favor of two more generic free functions that return adict
of data arrays that can then directly be passed to theplot
function:collapse(d, keep='x')
slices all dimensions away to keep only'x'
, thus always returning 1D slices.slices(d, dim='x')
slices along dimension'x'
, returning slices withndim-1
dimensions contaiing all dimensions other than'x'
.
Contributors#
Owen Arnold, David Fairbrother, Simon Heybrock, Daniel Nixon, Pawel Ptasznik, Piotr Rozyczko, and Neil Vaytet
v0.3 (March 2020)#
Many bug fixes and small additions
Multi-threading with TBB for many operations.
Performance improvements in hotspots
Remove
Dim
labels in favor of plain strings. Connected to this, thelabels
property for data arrays and datasets has been removed. Usecoords
instead.Start to support
out
arguments (not everywhere yet)scipp.neutron
: Instrument view added
Contributors in this release: Owen Arnold, Simon Heybrock, Daniel Nixon, Dimitar Tasev, and Neil Vaytet
v0.2 (December 2019)#
Support for masks stored in
DataArray
andDataset
.Support for
groupby
, implementing a split-apply-combine approach as known from pandas.Enhanced support for event data:
Histogramming with “weighted” data.
Multiplication/division operators between event data and histogram.
Enhanced plotting support:
Now focussing on
matplotlib
.Multi-dimensional plots with interactive sliders, and much more.
Significant performance improvements for majority of operations. Typically performance is now in the same ballpark as what the memory bandwidth on a single CPU core can support.
Fancy
_repr_html_
for quick views of datasets in Jupyter notebooks.Conda packages now also available for Windows.
scipp.neutron
gets improved converters from Mantid, supporting neutron monitors, sample information, and run information stored as attributes.
Contributors in this release: Owen Arnold, Igor Gudich, Simon Heybrock, Daniel Nixon, Dimitar Tasev, and Neil Vaytet
v0.1 (September 2019)#
This is the first official release of scipp
.
It is not yet meant for production-use, but marks a big step for us in terms of usability and features.
The API may change without notice in future releases.
Features:
All key data structures (
Variable
,DataArray
, andDataset
).Slicing.
Basic arithmetic operations.
Physical units.
Propagation of uncertainties.
Event data.
Limitations:
Limited performance and no parallelization.
Numerous “edge cases” not supported yet.
While tested, probably far from bug-free.
Contributing Organizations#
a European Spallation Source ERIC, Sweden
c Tessella, UK