Data handling¶
The data_handling
module provides functionality for loading and inspecting data.
Instances (objects) of the class DataContainer
are created by loading data from file.
Afterwards one can access, e.g., the geometry via the DataContainer.geometry
property, which is a Geometry
object.
The series of measurements (if available) is accessible via the DataContainer.projections
property, which is a ProjectionStack
object.
The latter acts as a list of individual measurements, which are provided as Projection
objects.
The mumott.data_handling.utilities
module provides convenience function for the calculation of transmittivities and absorbances from diode data.
- class mumott.data_handling.DataContainer(data_path=None, data_type='h5', skip_data=False, nonfinite_replacement_value=None)[source]¶
Instances of this class represent data read from an input file in a format suitable for further analysis. The two core components are
geometry
andprojections
. The latter comprises a list ofProjection
instances, each of which corresponds to a single measurement.By default all data is read, which can be rather time consuming and unnecessary in some cases, e.g., when aligning data. In those cases, one can skip loading the actual measurements by setting
skip_data
toTrue
. The geometry information and supplementary information such as the diode data will still be read.Example
The following code snippet illustrates the basic use of the
DataContainer
class.First we create a
DataContainer
instance, providing the path to the data file to be read.>>> from mumott.data_handling import DataContainer >>> dc = DataContainer('tests/test_full_circle.h5')
One can then print a short summary of the content of the
DataContainer
instance.>>> print(dc) ========================================================================== DataContainer -------------------------------------------------------------------------- Corrected for transmission : False ...
To access individual measurements we can use the
projections
attribute. The latter behaves like a list, where the elements of the list areProjection
objects, each of which represents an individual measurement. We can print a summary of the content of the first projection.>>> print(dc.projections[0]) -------------------------------------------------------------------------- Projection -------------------------------------------------------------------------- hash_data : 3f0ba8 hash_diode : 808328 hash_weights : 088d39 rotation : [1. 0. 0.], [ 0. -1. 0.], [ 0. 0. -1.] j_offset : 0.0 k_offset : 0.3 inner_angle : None outer_angle : None inner_axis : 0.0, 0.0, -1.0 outer_axis : 1.0, 0.0, 0.0 --------------------------------------------------------------------------
- Parameters
data_path (str, optional) – Path of the data file relative to the directory of execution. If None, a data container with an empty
projections
attached will be initialized.data_type (str, optional) – The type (or format) of the data file. Supported values are
h5
(default) for hdf5 format andNone
for an emptyDataContainer
that can be manually populated.skip_data (bool, optional) – If
True
, will skip data from individual measurements when loading the file. This will result in a functioninggeometry
instance as well asdiode
andweights
entries in each projection, butdata
will be empty.nonfinite_replacement_value (float, optional) – Value to replace nonfinite values (
np.nan
,np.inf
, and-np.inf
) with in the data, diode, and weights. IfNone
(default), an error is raised if any nonfinite values are present in these input fields.
- append(f)[source]¶
Appends a
Projection
to theprojections
attached to thisDataContainer
instance.- Return type
- correct_for_transmission()[source]¶
Applies correction from the input provided in the
diode
field. Should only be used if this correction has not been applied yet.- Return type
- property data: ndarray[Any, dtype[float64]]¶
The data in the
projections
object attached to thisDataContainer
instance.
- property diode: ndarray[Any, dtype[float64]]¶
The diode data in the
projections
object attached to thisDataContainer
instance.
- property projections: ProjectionStack¶
The projections, containing data and geometry.
- property weights: ndarray[Any, dtype[float64]]¶
The weights in the
projections
object attached to thisDataContainer
instance.
Utilities¶
- mumott.data_handling.utilities.get_absorbances(diode, **kwargs)[source]¶
Calculates the absorbance based on the transmittivity of the diode data.
Notes
The absorbance is defined as the negative base-10 logarithm of the transmittivity. Specifically,
\[A(i, j, k) = -\log_{10}(T(i, j, k))\]where \(T\) is the transmittivity, normalized to the open interval \((0, 1]\). It can be inferred from this formula why \(T(i, j, k)\) must not have values which are equal to or smaller than \(0\), as that would give a non-finite absorbance. Similarly, values greater than \(1\) would result in physically impossible negative absorbances.
The transmittivity is calculated directly from diode readouts, which may or may not already be normalized and clipped. When using already normalized diode values, it is best to set the keyword argument
normalization_percentile
to100
, and the argumentcutoff_values
to whatever cutoff your normalization used.See
get_transmittivities()
for more details on the transmittivity calculation.- Parameters
- Return type
- Returns
A dictionary with the absorbance, and the union of the cutoff masks from
get_transmittivities()
.
- mumott.data_handling.utilities.get_transmittivities(diode, normalize_per_projection=False, normalization_percentile=99.9, cutoff_values=(0.0001, 1.0))[source]¶
Calculates the transmittivity from the diode, i.e., the fraction of transmitted intensity relative to a high percentile.
Notes
Diode readouts may be given in various formats such as a count or a current. When doing absorption tomography, one is generally interested in the fraction of transmitted intensity. Since we do not generally have access to the incoming flux, or its theoretical readout at complete transmission, we can instead normalize the diode readout based on the largest values, where the beam has only passed through some air. We thus want to compute
\[T(i, j, k) = \frac{I_T(i, j, k)}{I_0}\]where \(I_T(i, j, k)\) is the diode readout value at projection \(i\), and pixel \((j, k)\) with the approximation \(I_0 \approx \text{max}(I_T(i, j, k))\).
To avoid routine normalization based on individual spurious readouts (from, e.g., hot pixels), by default the normalization is done based on the 99.9th percentile rather than the strict maximum. The normalized values are then clipped to the interval specified by
cutoff_values
, by default(1e-4, 1.0)
. A mask is returned which masks out any values outside this range, which can be useful to mask out spurious readouts.If the transmittivities are to be used to normalize SAXS data, one should leave the
normalize_per_projection
option atFalse
, because the SAXS data also scales with the incoming flux, and thus we want any variations in flux between projections to be preserved in the transmittivities.However, if the transmittivities are to be used for transmission (or absorption) tomography, then the
normalize_per_projection
option should be set toTrue
. Since we are interested in the transmittivity of the sample irrespective of the incoming flux, we are therefore better off assuming the flux is constant over each projection. This corresponds to the slightly modified computation\[T(i, j, k) = \frac{I_T(i, j, k)}{I_0(i)}\]with the approximation \(I_0(i) \approx \text{max}_{j, k}(I_T(i, j, k))\), with the understanding that we take the maximum value for each projection \(i\).
- Parameters
diode (
ndarray
[Any
,dtype
[TypeVar
(_ScalarType_co
, bound=generic
, covariant=True)]]) – An array of diode readouts.normalize_per_projection (
bool
) – IfTrue
, the diode will be normalized projection-wise. This is the appropriate choice for absorption tomography. For SAXS, it is preferable to normalize the diode across the entire set of diode measurements in order to account for possible variations in flux. Default value isFalse
.normalization_percentile (
float
) – The percentile of values in either the entire set of diode measurements or each projection (depending onnormalize_per_projection
) to use for normalization. The default value is99.9
. Values above this range will be clipped. If you are certain that you do not have spuriously large diode readout values, you can specify100.
as the percentile instead.cutoff_values (
Tuple
[float
,float
]) – The cutoffs to use for the transmittivity calculation. Default value is(1e-4, 1.0)
, i.e., one part in ten thousand for the lower bound, and a hundred percent for the upper bound. For values outside of this range, it may be desirable to mask them out during any calculation. For this purpose, acutoff_mask
is included in the return dictionary with the same shape as the weights inprojections
. In some cases, you may wish to specify other bounds. For example, if you know that your sample is embedded in a substrate which reduces the maximum possible transmittivity, you may wish to lower the upper bound. If you know that your sample has extremely low transmittivity (perhaps compensated with a very long exposure time), then you can set the lower cutoff even lower. The cutoffs must lie within the open interval(0, 1]
. A lower bound of0
is not permitted since this would lead to an invalid absorbance.
- Return type
- Returns
A dictionary with three entries,
transmittivity
,cutoff_mask_lower
, andcutoff_mask_upper
.