Versions Compared

Version Old Version 62 New Version 63
Changes made by

Kaz Wanelik

Kaz Wanelik

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

NxtomoLoader
Process categoryBrief description

Computational demand

for typical tomography data

(low, medium, high)

Comment(s)Reference(s)Common alternative process(es)Known issues
loaderTo specify the location of raw dataset(s) to be used as input.Low
  1. Implements lazy loading, i.e. no data is actually loaded until the first genuine access request is made by a subsequent process in the process list (as usual, the initial loading times depend on data chunking of source dataset(s)).
  2. Capable of handling the case of dark- and flat-field data being supplied in two individual NeXus datasets, one for the dark-field data and the other for the flat-field data (these two datasets are most often stored in two separate files, but it is also possible to have them stored as two separate datasets in a single file). No image-key information is required in this case to accompany either of the two datasets, and, if present (at the location specified by image_key_path), this information is ignored. This functionality can be used not only to reconstruct tomography data but also to process (e. g. dark-and-flat correct) radiography  data for which darks, flats and projections are typically provided in three separate files.
  3. Does not handle the case of the NXtomo entry pointing to a set of paths to individual TIFF files (as opposed to a single HDF5 dataset). 
  4. Specifying the location of dark- and flat-field data in NxtomoLoader is not sufficient for the dark-and-flat-field correction to be applied (DarkFlatFieldCorrection needs to be explicitly included in every process list that seeks this correction).
  1. NeXus NXtomo application definition
  2. NeXus file format
  3. HDF5 data model & file format


  1. Currently, NxtomoLoader can only handle either the case of both dark- and flat-field data being supplied in the same (combined) dataset as sample projections or that of both being provided in two separate external datasets, one containing only darks and the other only flats. For instance, NxtomoLoader will currently fail if darks are supplied together with sample projections, whereas flats are provided in a separate external dataset:
    Jira Legacy
    serverDiamond JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverId67144b25-dc51-3dfe-84e1-c04202075171
    keySCI-6814
         


...

ItemParameter nameParameter formatExample(s)Comment(s)
Parameter valueEffect
1preview[ind_0, ind_1,... ind_(n-1)]


[ ]To select the entire n-dimensional dataset of sample projections, stored at location specified by the data_path parameter and accompanied by the image-key data supplied via the image_key_path parameter.
  1. The preview parameter provides a convenient mechanism for restricting the entire dataset to a desired subset.
  2. The default value of the preview parameter, [ ] (empty Python list), represents the entire dataset.
[:, :, :]To select the entire 3d dataset of sample projections, stored at location specified by the data_path parameter and accompanied by the image-key data supplied via the image_key_path parameter.
[:, s, :] To select from a given 3d dataset of sample projections (stored at the data_path location with the corresponding image keys at the image_key_path location) only a subset of data corresponding to the s-th sinogram.This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
[p, :, :]To select from a given 3d dataset of sample projections (stored at data_path location with the corresponding image keys at the image_key_path location) only a subset of data corresponding to the p-th sample projection.This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
[:, :, 12:-34]To select from a given 3d dataset of sample projections (stored at the data_path location with the corresponding image keys at the image_key_path location) only a subset of data generated by cropping the width of each sample projection by 12 pixels from the left-hand side and 34 pixels from the right-hand side.This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
[:, mid - 1:mid + 1, :]To select from a given 3d dataset of sample projections (stored at the data_path location with the corresponding image keys at the image_key_path location) only a subset of data corresponding to the middle 2 sinograms.mid is one of Savu keywords (with an obvious meaning).

[0:end:2, :, :]

[::2, :, :]

To select from a given 3d dataset of sample projections (stored at the data_path location with the corresponding image keys at the image_key_path location) only a subset of data corresponding to every other sample projection.end is one of Savu keywords (with an obvious meaning).

[0:end:end, :, :]

[::end, :, :]

To select from a given 3d dataset of sample projections (stored at the data_path location with the corresponding image keys at the image_key_path location) only the first and the last sample projection.

[:, 0:end:2, :]

[:, ::2, :]

To select from a given 3d dataset of sample projections (stored at the data_path location with the corresponding image keys at the image_key_path location) only a subset of data corresponding to every other sinogram.
2

image_key_path




  1. Mandatory dataset (can be set to None for radiography images, if dark- and flat-field images are provided in two separate scan or data files, respectively).in the same image dataset as sample projections. If dark- and flat-field images are not stored in the same dataset as sample projections, then this image_key_path parameter is ignored and can be set to None. 

  2. The values of image keys are expected to adhere to the following convention: 2 for dark-field image; 1 for flat-field image; 0 for sample projection.

3

name


<any-string-of-characters>To label the principal image dataset with a meaningful name, provided by the user ('tomo' by default). If desired, this name can then be used to refer to this dataset in any subsequent processes in a given process list (Savu automatically propagates this name wherever it is required in a given process list, provided no other name is entered in those places by the user). For example, if the name parameter were to be genuinely set to 'any-string-of-characters', then the reconstructed volume would have been found at /entry/final_result_any-string-of-characters/data in the principal output file, named scan-number_processed.nxs, and, if present, TiffSaver would have saved the output TIFF images in a directory called 'TiffSaver-any-string-of-characters'.

radiographyThis is an example of a string that may be more appropriate than 'tomo' for naming an image dataset collected, for example, at a single angle (as opposed to a series of angles over the range of 180 deg, used in tomography). 
4

3d_to_4d



Can be used to load a series of individual tomography datasets from a single multi-tomography dataset.  
5

flat

[<path-to-NeXus-file>, <path-to-dataset>, <exposure-time-compensation-factor>][/dls/i13/data/2018/cm123-4/raw/5678.nxs, entry1/instrument/detector/pco1/data, 1.25]To select all flat-field images from the flats-only dataset located at entry1/instrument/detector/pco1/data inside a NeXus scan file, stored on the file system at /dls/i13/data/2018/cm123-4/raw/5678.nxs, and then to multiply their pixel values by 1.25 (to compensate for the flat-field exposure interval being 0.8 times shorter than the sample-projection exposure time).
  1. This parameter needs to be specified only if flats are not stored in the same dataset as sample projections.
  2. If <path-to-NeXus-file> points to an accessible NeXus file, then flats are loaded from a dataset found at <path-to-dataset> inside that file, and their pixel values are then multiplied by <exposure-time-compensation-factor>.
  3. The dataset at <path-to-dataset> must contain only flats. These external flats override any internal flats that may be stored alongside sample projections in the principal raw-data NeXus scan file.
  4. No image-key information is required to accompany the flat dataset, and, if present (at the location specified by image_key_path), this information is ignored.
  5. Note that, if the flat-field dataset is integer-valued, then the multiplication by <exposure-time-compensation-factor> (even if the latter is equal to 1) changes this data type to the floating-point type, which in turn may be unsuitable as input for the immediately succeeding process, e.g. Dezinger (use DezingerSinogram instead).
6

data_path




Mandatory dataset.
7

dark

[<path-to-NeXus-file>, <path-to-dataset>, <exposure-time-compensation-factor>][/dls/i13/data/2018/cm123-4/raw/8765.nxs, entry1/instrument/detector/pco1/data, 1]To select all dark-field images from the darks-only dataset located at entry1/instrument/detector/pco1/data inside a NeXus scan file, stored on the file system at /dls/i13/data/2018/cm123-4/raw/8765.nxs, and then to multiply their pixel values by 1 (this trivial multiplication factor should be used whenever the dark-field exposure interval is the same as the sample-projection exposure interval, in which case no change to recorded intensities is, of course, necessary).
  1. This parameter needs to specified only if darks are not stored in the same dataset as sample projections.
  2. If <path-to-NeXus-file> points to an accessible NeXus file, then flats are loaded from a dataset found at <path-to-dataset> inside that file, and their pixel values are then multiplied by <exposure-time-compensation-factor>.
  3. The dataset at <path-to-dataset> must contain only darks. These external darks override any internal darks that may be stored alongside sample projections in the principal raw-data NeXus scan file.
  4. No image-key information is required to accompany the dark dataset, and, if present (at the location specified by image_key_path), this information is ignored.
  5. Note that, if the dark-field dataset is integer-valued, then the multiplication by <exposure-time-compensation-factor> (even if the latter is equal to 1) changes this data type to the floating-point type, which in turn may be unsuitable as input for the immediately succeeding process, e.g. Dezinger (use DezingerSinogram instead).
8

angles


None (Python keyword)If the angles parameter is set to None, then this loader attempts to access values in a default dataset, located at entry1/tomo_entry/data/rotation_angle in the input NeXus scan file.

numpy.linspace(0,180,1801)

array([  0.00000000e+00,   1.00000000e-01,   2.00000000e-01, ...,

         1.79800000e+02,   1.79900000e+02,   1.80000000e+02])

9

ignore_flats

[ind_1, ind_2,... ind_m][1, r]To ignore the initial and the r-th batch in a given series of batches of flats (each containing however many individual flat-field images), stored in the same dataset as darks and sample projections.
  1. Useful for excluding compromised flats in combined datasets containing multiple batches of interspaced flats.
  2. In the ignore_flats list, batch indexing starts from 1 (not 0).

...