NxtomoLoader

Summary

NxtomoLoader

Process category

Brief description

Computational demand

for typical tomography data

(low, medium, high)

Comment(s)

Reference(s)

Common alternative process(es)

Known issues

loader

To specify the location of raw dataset(s) to be used as input.

Low

Implements lazy loading, i.e. no data is actually loaded until the first genuine access request is made by a subsequent process in the process list (as usual, the initial loading times depend on data chunking of source dataset(s)).
Capable of handling the case of dark- and flat-field data being supplied in two individual NeXus datasets, one for the dark-field data and the other for the flat-field data (these two datasets are most often stored in two separate files, but it is also possible to have them stored as two separate datasets in a single file). No image-key information is required in this case to accompany either of the two datasets, and, if present (at the location specified by image_key_path), this information is ignored. This functionality can be used not only to reconstruct tomography data but also to process (e. g. dark-and-flat correct) radiography data for which darks, flats and projections are typically provided in three separate files.
Does not handle the case of the NXtomo entry pointing to a set of paths to individual TIFF files (as opposed to a single HDF5 dataset).
Specifying the location of dark- and flat-field data in NxtomoLoader is not sufficient for the dark-and-flat-field correction to be applied (DarkFlatFieldCorrection needs to be explicitly included in every process list that seeks this correction).

Currently, NxtomoLoader can only handle either the case of both dark- and flat-field data being supplied in the same (combined) dataset as sample projections or that of both being provided in two separate external datasets, one containing only darks and the other only flats. For instance, NxtomoLoader will currently fail if darks are supplied together with sample projections, whereas flats are provided in a separate external dataset:
Error rendering macro 'jira' : null

Parameters

Brief description

Savu Configurator command

>>> disp -avv

-------------------------------------------------------------------------------------
 1) NxtomoLoader(savu.plugins.loaders.full_field_loaders.nxtomo_loader)              
  A class for loading standard tomography data in Nexus format.                      
    1)                preview : []
    A slice list of required frames.
    2)         image_key_path : entry1/tomo_entry/instrument/detector/image_key
    Path to the image key entry inside the nxs file.
    3)                   name : tomo
    The name assigned to the dataset.
    4)               3d_to_4d : False
    Set to true if this reshape is required.
    5)                   flat : [None, None, 1]
    Optional Path to the flat field data file, nxs path and scale value.
    6)              data_path : entry1/tomo_entry/data/data
    Path to the data inside the file.
    7)                   dark : [None, None, 1]
    Optional path to the dark field data file, nxs path and scale value.
    8)                 angles : None
    A python statement to be evaluated or a file.
    9)           ignore_flats : None
    List of batch numbers of flats (start at 1) to ignore.
-------------------------------------------------------------------------------------

>>>

Additional notes

For basic information on this process, please use the disp -av (or disp -avv or disp -v[v] <process index>) command in Savu Configurator (see above). The table below is intended to provide some additional notes on a number of selected topics:

Item	Parameter name	Parameter format	Example(s)		Comment(s)
Item	Parameter name	Parameter format	Parameter value	Effect	Comment(s)
1	preview	[ind_0, ind_1,... ind_(n-1)]	[ ] (default)	To select the entire n-dimensional dataset of sample projections, stored at location specified by the *data_path* parameter and accompanied by the image-key data supplied via the *image_key_path* parameter.	The *preview* parameter provides a convenient mechanism for restricting the entire dataset to a desired subset. The default value of the *preview* parameter, [ ] (empty Python list), represents the entire dataset.
			[:, :, :]	To select the entire 3d dataset of sample projections, stored at location specified by the *data_path* parameter and accompanied by the image-key data supplied via the *image_key_path* parameter.
			[:, s, :]	To select from a given 3d dataset of sample projections (stored at the *data_path* location with the corresponding image keys at the *image_key_path* location) only a subset of data corresponding to the s-th sinogram.	This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
			[p, :, :]	To select from a given 3d dataset of sample projections (stored at *data_path* location with the corresponding image keys at the *image_key_path* location) only a subset of data corresponding to the p-th sample projection.	This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
			[:, :, 12:-34]	To select from a given 3d dataset of sample projections (stored at the *data_path* location with the corresponding image keys at the *image_key_path* location) only a subset of data generated by cropping the width of each sample projection by 12 pixels from the left-hand side and 34 pixels from the right-hand side.	This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
			[:, mid - 1:mid + 1, :]	To select from a given 3d dataset of sample projections (stored at the *data_path* location with the corresponding image keys at the *image_key_path* location) only a subset of data corresponding to the middle 2 sinograms.	mid is one of Savu keywords (with an obvious meaning).
			[0:end:2, :, :] [::2, :, :]	To select from a given 3d dataset of sample projections (stored at the *data_path* location with the corresponding image keys at the *image_key_path* location) only a subset of data corresponding to every other sample projection.	end is one of Savu keywords (with an obvious meaning).
			[0:end:end, :, :] [::end, :, :]	To select from a given 3d dataset of sample projections (stored at the *data_path* location with the corresponding image keys at the *image_key_path* location) only the first and the last sample projection.
			[:, 0:end:2, :] [:, ::2, :]	To select from a given 3d dataset of sample projections (stored at the *data_path* location with the corresponding image keys at the *image_key_path* location) only a subset of data corresponding to every other sinogram.
2	image_key_path		entry1/tomo_entry/instrument/detector/image_key (default)		Mandatory dataset if dark- and flat-field images are provided in the same image dataset as sample projections. If dark- and flat-field images are not stored in the same dataset as sample projections, then this *image_key_path* parameter is ignored and can be set to None. The values of image keys are expected to adhere to the following convention: 2 for dark-field image; 1 for flat-field image; 0 for sample projection.
3	name		<any-string-of-characters>	To label the loaded image dataset with a meaningful name, provided by the user ('tomo' by default). If desired, this name can then be used to refer to this dataset in the in_datasets parameter of any subsequent processes in a given process list. If no name is supplied in these subsequent in_datasets by the user, then Savu automatically propagates the name from NxtomoLoader as appropriate). For example, if the *name* parameter were to be genuinely set to 'any-string-of-characters', then the reconstructed volume would have been found at /entry/final_result_any-string-of-characters/data in the top-level output file, named scan-number_processed.nxs, and, if present, TiffSaver would have saved the output TIFF images in a directory called 'TiffSaver-any-string-of-characters'.
3	name		radiography	This is an example of a string that may be more appropriate than 'tomo' for naming an image dataset collected, for example, at a single angle (as opposed to a series of angles over the range of 180 deg, used in tomography).
4	3d_to_4d			Can be used to load a series of individual tomography datasets from a single multi-tomography dataset.
5	flat	[<path-to-NeXus-file>, <path-to-dataset>, <exposure-time-compensation-factor>] where <exposure-time-compensation-factor> = <sample-exposure-time>/<flat-exposure-time>	[/dls/i13/data/2018/cm123-4/raw/5678.nxs, entry1/instrument/detector/pco1/data, 1.25]	To select all flat-field images from the flats-only dataset located at entry1/instrument/detector/pco1/data inside a NeXus scan file, stored on the file system at /dls/i13/data/2018/cm123-4/raw/5678.nxs, and then to multiply their pixel values by 1.25 (to compensate for the flat-field exposure interval being 0.8 times shorter than the sample-projection exposure time).	This parameter needs to be specified only if flats are not stored in the same dataset as sample projections. If <path-to-NeXus-file> points to an accessible NeXus file, then flats are loaded from a dataset found at <path-to-dataset> inside that file, and their pixel values are then multiplied by <exposure-time-compensation-factor>. The dataset at <path-to-dataset> must contain only flats. These external flats override any internal flats that may be stored alongside sample projections in the principal raw-data NeXus scan file. When overriding internal flats, it is usually necessary to use the *angles* parameter (see below) to specify only the angles at which sample projections were collected (otherwise there will be a mismatch between the length the image dataset (stored by default in entry1/tomo_entry/data/data) and that of the angle dataset (stored by default in entry1/tomo_entry/data/rotation_angle). No image-key information is required to accompany the flat dataset, and, if present (at the location specified by image_key_path), this information is ignored. Note that, if the flat-field dataset is integer-valued, then the multiplication by <exposure-time-compensation-factor> (even if the latter is equal to 1) changes this data type to the floating-point type, which in turn may be unsuitable as input for the immediately succeeding process, e.g. Dezinger (use DezingerSinogram instead).
6	data_path		entry1/tomo_entry/data/rotation_angle (default)		Mandatory dataset.
7	dark	[<path-to-NeXus-file>, <path-to-dataset>, <exposure-time-compensation-factor>] where <exposure-time-compensation-factor> = <sample-exposure-time>/<dark-exposure-time>	[/dls/i13/data/2018/cm123-4/raw/8765.nxs, entry1/instrument/detector/pco1/data, 1]	To select all dark-field images from the darks-only dataset located at entry1/instrument/detector/pco1/data inside a NeXus scan file, stored on the file system at /dls/i13/data/2018/cm123-4/raw/8765.nxs, and then to multiply their pixel values by 1 (this trivial multiplication factor should be used whenever the dark-field exposure interval is the same as the sample-projection exposure interval, in which case no change to recorded intensities is, of course, necessary).	This parameter needs to specified only if darks are not stored in the same dataset as sample projections. If <path-to-NeXus-file> points to an accessible NeXus file, then flats are loaded from a dataset found at <path-to-dataset> inside that file, and their pixel values are then multiplied by <exposure-time-compensation-factor>. The dataset at <path-to-dataset> must contain only darks. These external darks override any internal darks that may be stored alongside sample projections in the principal raw-data NeXus scan file. When overriding internal darks, it is usually necessary to use the *angles* parameter (see below) to specify only the angles at which sample projections were collected (otherwise there will be a mismatch between the length the image dataset (stored by default in entry1/tomo_entry/data/data) and that of the angle dataset (stored by default in entry1/tomo_entry/data/rotation_angle). No image-key information is required to accompany the dark dataset, and, if present (at the location specified by image_key_path), this information is ignored. Note that, if the dark-field dataset is integer-valued, then the multiplication by <exposure-time-compensation-factor> (even if the latter is equal to 1) changes this data type to the floating-point type, which in turn may be unsuitable as input for the immediately succeeding process, e.g. Dezinger (use DezingerSinogram instead).
8	angles		None (Python keyword)	If the *angles* parameter is set to None, then this loader attempts to access values in a default dataset, located at entry1/tomo_entry/data/rotation_angle in the input NeXus scan file.
8	angles		numpy.linspace(0,180,1801)	array([ 0.00000000e+00, 1.00000000e-01, 2.00000000e-01, ..., 1.79800000e+02, 1.79900000e+02, 1.80000000e+02])
9	ignore_flats	[ind_1, ind_2,... ind_m]	[1, r]	To ignore the initial and the r-th batch in a given series of batches of flats (each containing however many individual flat-field images), stored in the same dataset as darks and sample projections.	Useful for excluding compromised flats in combined datasets containing multiple batches of interspaced flats. In the ignore_flats list, batch indexing starts from 1 (not 0).

Usage

How to load dark- and flat-field data from individual NeXus datasets?

Any standard tomography NeXus scan file, generated by GDA, stores all the recorded images (i.e. sample projections as well as dark- and flat-field images) in a single 3-dimensional dataset located at entry1/tomo_entry/data/data. This combined image dataset is always accompanied by another important dataset which contains the corresponding image-key information, located in the same file at entry1/tomo_entry/instrument/detector/image_key. For obvious reasons, the default values of NxtomoLoader's parameters have been selected to reflect this standard scenario. If necessary, however, NxtomoLoader can also be used to load dark- and flat-field data from two separate NeXus datasets, one containing only darks and the other only flats (these two datasets can be provided in the same NeXus file or in two separate NeXus files, one for darks and the other for flats). This non-standard scenario can be handled by making appropriate modifications to the default settings of the flat and dark parameters (see the table above for more details).

Tomography Reconstruction

NxtomoLoader

Analytics

Summary

Parameters

Brief description

Additional notes

Usage

How to load dark- and flat-field data from individual NeXus datasets?

Related content