Versions Compared

Version Old Version 47 New Version 48
Changes made by

Kaz Wanelik (Unlicensed)

Kaz Wanelik (Unlicensed)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.


Table of Contents

Summary


NxtomoLoader
Process categoryBrief description

Computational demand

for typical tomography data

(low, medium, high)

Comment(s)Reference(s)Common alternative process(es)Known issues
loaderTo specify the location of raw dataset(s) to be used as input.Low
  1. Implements lazy loading, i.e. no data is actually loaded until the first genuine access request is made by a subsequent process in the process list (as usual, the initial loading times depend on data chunking of source dataset(s)).
  2. Capable of handling the case of dark- and flat-field data being supplied in two individual NeXus datasets, one for the dark-field data and the other for the flat-field data (these two datasets are most often stored in two separate files, but it is also possible to have them stored as two separate datasets in a single file). No image-key information is required in this case to accompany either of the two datasets, and, if present (at the location specified by image_key_path), this information is ignored.
  3. Does not handle the case of the NXtomo entry pointing to a set of paths to individual TIFF files (as opposed to a single HDF5 dataset). 
  4. Specifying the location of dark- and flat-field data in NxtomoLoader is not sufficient for the dark-and-flat-field correction to be applied (DarkFlatFieldCorrection needs to be explicitly included in every process list that seeks this correction).
  1. NeXus NXtomo application definition
  2. NeXus file format
  3. HDF5 data model & file format



  1. Currently, NxtomoLoader can only handle either the case of both dark- and flat-field data being supplied in the same (combined) dataset as sample projections or that of both being provided in two separate datasets, one containing only darks and the other only flats. For instance, NxtomoLoader will currently fail if flats are supplied together with sample projections, whereas darks are provided in a separate external dataset;
    Jira Legacy
    serverDiamond JIRA
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverId67144b25-dc51-3dfe-84e1-c04202075171
    keySCI-6814
         



Parameters


Brief description

...

ItemParameter nameParameter formatExample(s)Comment(s)
Parameter valueEffect
1preview[ind_0, ind_1,... ind_(n-1)]


[ ]To load select the entire n-dimensional dataset of sample projections, stored at location specified by the data_path parameter and accompanied by the image-key data supplied by the image_key_path parameter.Empty Python list, [ ], is the default
  1. The preview parameter provides a convenient mechanism for restricting the entire dataset to a desired subset.
  2. The default default value of the preview parameter, [ ] (empty Python list), represents the entire dataset.
[:, :, :]To load select the entire 3d dataset of sample projections, stored at location specified by the data_path parameter and accompanied by the image-key data supplied by the image_key_path parameter.
[:, s, :] To load select from a given 3d dataset of sample projections (stored at the data_path location with image keys at the image_key_path location) only a subset of data corresponding to the s-th sinogram.This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
[p, :, :]To load select from a given 3d dataset of sample projections (stored at data_path location with image keys at the image_key_path location) only a subset of data corresponding to the p-th sample projection.This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
[:, :, 123:-123]To load select from a given 3d dataset of sample projections (stored at the data_path location with image keys at the image_key_path location) only a subset of data generated by cropping the width of each sample projection by 123 pixels on either side.This particular interpretation applies if data are laid out in the (angle_enumeration, image_height, image_width) format.
[:, mid - 1:mid + 1, :]To load select from a given 3d dataset of sample projections (stored at the data_path location with image keys at the image_key_path location) only a subset of data corresponding to the middle 2 sinograms.mid is one of Savu keywords (with an obvious meaning).

[0:end:2, :, :]

[::2, :, :]

To load select from a given 3d dataset of sample projections (stored at the data_path location with image keys at the image_key_path location) only a subset of data corresponding to every other sample projection.end is one of Savu keywords (with an obvious meaning).

[0:end:end, :, :]

[:, ::end, :]

To select from a given 3d dataset of sample projections (stored at the data_path location with image keys at the image_key_path location) only the first and the last sample projection.

[:, 0:end:2, :]

[:, ::2, :]

To load select from a given 3d dataset of sample projections (stored at the data_path location with image keys at the image_key_path location) only a subset of data corresponding to every other sinogram.
2

image_key_path




  1. Mandatory dataset.

  2. The values of image keys are as follows: 2 represents dark-field image; 1 represents flat-field image; 0 represents sample projection.

3

name





4

3d_to_4d





5

flat

[<path-to-NeXus-file>, <path-to-dataset>, <exposure-time-compensation-factor>][/dls/i13/data/2018/cm123-4/raw/55678.nxs, entry1/instrument/detector/pco1/data, 1.25]To load all flats select all flat-field images from the flats-only dataset found located at entry1/instrument/detector/pco1/data inside a NeXus scan file, located stored on the file system at /dls/i13/data/2018/cm123-4/raw/55678.nxs, and then to multiply their pixel values by 1.25 (to compensate for the flat-field exposure time interval being 0.8 times shorter than the sample-projection exposure time).
  1. Specify this parameter This parameter needs to be specified only if flats are not stored in the same dataset as sample projections.
  2. If <path-to-NeXus-file> is not None, then flats are loaded from a dataset found at <path-to-dataset> inside a NeXus file, located on the file-system at <path-to-NeXus-file>, and their pixel values are then multiplied by <exposure-time-compensation-factor>.
  3. No image-key information is required to accompany the flat dataset, and, if present (at the location specified by image_key_path), this information is ignored.
  4. Note that, if the flat-field dataset is integer-valued, then the multiplication by <exposure-time-compensation-factor> (even if the latter is equal to 1) changes this data type to the floating-point type, which in turn may be unsuitable as input for the immediately succeeding process, e.g. Dezinger (use DezingerSinogram instead).
6

data_path




Mandatory dataset.
7

dark

[<path-to-NeXus-file>, <path-to-dataset>, <exposure-time-compensation-factor>][/dls/i13/data/2018/cm123-4/raw/68765.nxs, entry1/instrument/detector/pco1/data, 1]To load all darks select all dark-field images from the darks-only dataset found located at entry1/instrument/detector/pco1/data inside a NeXus scan file, located stored on the file system at /dls/i13/data/2018/cm123-4/raw/68765.nxs, and then to multiply their pixel values by 1 (this multiplication factor needs to be used whenever the dark-field exposure interval is the same as the sample-projection interval, in which case no change to recorded intensities is, of course, necessary).
  1. Specify this This parameter needs to specified only if darks are not stored in the same dataset as sample projections.
  2. If <path-to-NeXus-file> is not None, then darks are loaded from a dataset found at <path-to-dataset> inside a NeXus file, located on the file-system at <path-to-NeXus-file>, and their pixel values are then multiplied by <exposure-time-compensation-factor>.
  3. No image-key information is required to accompany the dark dataset, and, if present (at the location specified by image_key_path), this information is ignored.
  4. Note that, if the dark-field dataset is integer-valued, then the multiplication by <exposure-time-compensation-factor> (even if the latter is equal to 1) changes this data type to the floating-point type, which in turn may be unsuitable as input for the immediately succeeding process, e.g. Dezinger (use DezingerSinogram instead).
8

angles


None (Python keyword)If the angles parameter is set to None, then this loader attempts to find them access values in a default dataset, located at entry1/tomo_entry/data/rotation_angle in the input NeXus scan file.
numpy.linspace(0,180,1801)

array([  0.00000000e+00,   1.00000000e-01,   2.00000000e-01, ...,

         1.79800000e+02,   1.79900000e+02,   1.80000000e+02])


9

ignore_flats

[ind_1, ind_2,... ind_m][1, r]To ignore the initial and the r-th batch in a given series of batches of flats (each containing however many individual flat-field images), stored in the same dataset as darks and sample projections.
  1. Useful for excluding compromised flats for scans in combined datasets containing multiple batches of mid-scan interspaced flats.
  2. In the ignore_flats list, batch indexing starts from 1 (not 0).

...

Any standard tomography NeXus scan file, generated by GDA, stores all the recorded images (i.e. sample projections as well as dark- and flat-field images) in a single 3-dimensional dataset located at entry1/tomo_entry/data/data. This combined image dataset is always accompanied by another important dataset which contains the corresponding image-key information, located in the same file at entry1/tomo_entry/instrument/detector/image_key. The For obvious reasons, the default values of NxtomoLoader's parameters have been selected to reflect this standard scenario. HoweverIf necessary, however, NxtomoLoader can also be used to load dark- and flat-field data from two individual NeXus datasets, if necessaryseparate NeXus datasets, one containing only darks and the other only flats (these two datasets can be stored in the same NeXus file or in two separate NeXus files, one for darks and the other for flats). This non-standard scenario can be handled by making appropriate modifications to the default settings of the flat and dark parameters (see the table above for more details).

...