...
The width, length and bit depth of TIFF images can be determined in many different ways. For example, the Linux command tiffinfo can be used to print information about the input TIFF file(s) in the following way:Anchor A1 A1 tiffinfo [options] input1.tif input2.tif...
For example,
Code Block title Linux command tiffinfo 06029.tif
displays the following summary information:
Code Block title Linux output from tiffinfo TIFFReadDirectory: Warning, 06029.tif: unknown field with tag 65000 (0xfde8) encountered. TIFF Directory at offset 0x146d308 (21418760) Image Width: 4008 Image Length: 2672 Bits/Sample: 16 Sample Format: unsigned integer Compression Scheme: None Photometric Interpretation: min-is-black Samples/Pixel: 1 Rows/Strip: 2672 Planar Configuration: single image plane Make: Unknown Model: Unknown Tag 65000: 727733745.824557
It is sometimes desirable to reconstruct a sub-set, or a slab, of slices rather than the complete set. The recon_arrayxml.py script is capable of accomplishing this task with the help of the F- and L-options.Anchor A2 A2
For example,Code Block title Linux command python2.6 /dls_sw/i13/software/gda/config/tomography_scripts/recon_arrayxml.py -I settings.xml -o ../../reconstruction/564/ -C 2000.5 -F 1299 -L 1300
will reconstruct a slab consisting of exactly 1 slice (=1300-1299), namely that indexed 1299.
The ROI element included in your settings.xml file needs to be modified in the following way:Anchor A3 A3 <ROI>
<Type info="Standard, Rectangle">Rectangle</Type>
<Xmin>580</Xmin>
<Xmax>3000</Xmax>
<Ymin>1225</Ymin>
<Ymax>1725</Ymax>
<OutputWidthType info="Standard, Given">Standard</OutputWidthType>
<OutputWidth>500</OutputWidth>
<Angle>0.0</Angle></ROI>
where the above values of Xmin, Xmax, Ymin, and Ymax need to be changed to reflect your particular reconstruction needs. Incidentally, these ROI coordinates can be determined, for example, by viewing the original projection images in ImageJ.
The information about all current SGE jobs can be obtained with the help of the qstat command. The state of a successfully submitted job is indicated by the status code qw or r.If a job encountered a problem, then its status code is Eqw or Rr. The possible status codes are:Anchor A4 A4
d(eletion)E(rror)
h(old)
r(unning)
R(estarted)
s(uspended)
S(uspended)
t(ransferring)
T(hreshold)
w(aiting)
For a more detailed explanation of all possible job states, see the table displayed below:
Category SGE Letter Code Description Pending qw Job is in the queue and waiting to be executed. hqw Job is in the queue and waiting to be executed but it is not elligible for execution due to a hold request or because it is waiting for completion of all the jobs in its job dependecies. hRqw Job is in the queue and waiting to be executed after being Restarted but it is not elligible for execution due to a hold request or because it is waiting for completion of all the jobs in its job dependecies.
Running r Job is running. t Job is transferring and about to be executed. Rr Job is running after being Restarted. Rt Job is transferring after being Restarted. Suspended s
ts
Job is suspended. S
tS
The queue containing the job is Suspended and therefore the job is also suspended. T
tT
At least one suspend Threshold of the queue containing the job has been exceeded and therefore the job has been suspended as a consequence. Rs
Rts
RS
RtS
RT
RtT
Job is in the Suspended category after being Restarted. Error Eqw
Ehqw
ERqw
Job in the Pending category is in the Error state due to its job properties and therefore could not be executed. Deletion dr
dt
dRr
dRt
ds
dts
dS
dtS
dT
dtT
dRs
dRts
dRS
dRtS
dRT
dRtT
Job in the Running or Suspended category which has been marked for deletion via the qdel command. If a SGE job is in the Eqw state, then the following command:
Code Block title Linux command qstat -j job_id | grep "error reason"
can be used to diagnose the errors (please replace the job_id argument with the identifier of your particular job).
The owner of a SGE job can delete it with the help of the qdel command:Anchor A5 A5 Code Block title Linux command qdel job_id
The nxs2tiff command can be used to extract TIFF images from a given HDF5 dataset accompanied by a NeXus scan file. One can use this command to extract all or any subset of images from this dataset.Anchor A6 A6 Code Block title Linux commands module load tomography; nxs2tiff -h
outputs a brief description of the command's arguments:
Code Block title Linux output >nxs2tiff -h Welcome to the DLS compute cluster For MPI jobs, please use 'module load openmpi'. If using a different OpenMPI installation, or manually specifying path to OpenMPI, option '-mca orte_forward_job_control 1' must be added to mpirun to ensure cluster functionality. Please report any issues to linux.manager@diamond.ac.uk Loading 64-bit scipy, version 0.10.0 /dls_sw/apps/tomopy/tomopy/src/tifffile.py:2170: UserWarning: failed to import _tifffile.decodepackbits warnings.warn("failed to import %s" % module_function) /dls_sw/apps/tomopy/tomopy/src/tifffile.py:2170: UserWarning: failed to import _tifffile.decodelzw warnings.warn("failed to import %s" % module_function) /dls_sw/apps/tomopy/tomopy/src/tifffile.py:2170: UserWarning: failed to import _tifffile.unpackints warnings.warn("failed to import %s" % module_function) Usage: nxs2tiff.py [options] file output_folder Options: --version show program's version number and exit -h, --help show this help message and exit -b BEGIN, --begin=BEGIN projection to begin from -s SLICES, --slices=SLICES number of projections to process -d DATASET, --dataset=DATASET Select the dataset to read from -k KEYS, --image_keys=KEYS Select the image_keys dataset -i DIM, --slice_dim=DIM Dimension to slice the data in to create the projections -f FORMAT, --format=FORMAT format for the output files -r, --robert output for robert --scan=SCAN scan number if needed (for roberts extraction) >
The simplest way to use nxs2tiff is to follow the following 3-argument pattern:Code Block title Linux command nxs2tiff -d /nexus/internal/path/to/dataset /path/to/nexus/file /path/to/output/directory
Please note that the above command requires the use of absolute paths. The default filename format for all extracted files is: p_%05d.tif (c.f. the f-option above).
Please also note that the default value for the d-option is /entry1/tomo_entry/instrument/detector/data, so there is normally no need to include this option unless the tomo entry is absent from your Nexus scan file, or is malformed, or if you would like to extract images from a dataset stored in a location different from this default path.
For example, the command given below extracts all images (flat fields, dark fields, and sample projections) in the TIFF format, and then saves the entire sequence of these images in a common directory:
Code Block title Linux command nxs2tiff -d /entry1/instrument/pco1_hw_hdf/data /dls/i13/data/2013/cm5937-3/raw/23912.nxs /dls/i13/data/2013/cm5937-3/processing/raw/23912/projections
Please note that the first input path, corresponding to the d (standing for data or dataset) option, is not a path to an HDF5 file but an internal path to a desired dataset inside your NeXus scan file (incidentally, the path to the latter file should be used as the second argument). This internal path can be verified in the usual way using hdfview:
If your Nexus scan file contains the relevant image-key data, then it is possible to extract and save your images in 3 separate directories depending on whether they are flat fields, dark fields, or sample projections. This can be done with the help of the r and scan options. For example, the command given below:
Code Block title Linux command nxs2tiff -r --scan 23912 /dls/i13/data/2013/cm5937-3/raw/23912.nxs /dls/i13/data/2013/cm5937-3/processing/raw
extracts sample-projections, flat-field (ff), and dark-field (df) images to <output_folder>/<scan>/projections, <output_folder>/ff1<scan>/projections and <output_folder>/df<scan>/projections, respectively.
Image type Image path relative to <output_folder> Image absolute path Comment(s) sample projection/23912/projections/dls/i13/data/2013/cm5937-3/processing/raw/23912/projectionsIf default format, p_%05d.tif, is used, filenames of extracted projection images are indexed from 0, e.g. p_00000.tif, p_00001.tif, p_00002.tif, etc. flat field/ff123912/projections/dls/i13/data/2013/cm5937-3/processing/raw/ff123912/projectionsdark field
/2013/cm5937-3/processing/raw/df23912/projectionsIf default format, p_%05d.tif, is used, filenames of extracted flat-field images are indexed from 0, e.g. p_00000.tif, p_00001.tif, p_00002.tif, etc. dark field/df23912/projections
/dls/i13/data/2013/cm5937-3/processing/raw/df23912/projectionsIf default format, p_%05d.tif, is used, filenames of extracted dark-field images are indexed from 0, e.g. p_00000.tif, p_00001.tif, p_00002.tif, etc. Please also note that the default value for the k-option is /entry1/tomo_entry/instrument/detector/image_key, so there is normally no need to include this option unless the tomo entry is absent from your Nexus scan file, or is malformed, or if you would like to utilise image keys stored in a location different from this default path.
To extract a subset of images, please use the b and s options.
When the nxs2tiff command is executed, the terminal output should contain the following message:
Code Block title Linux output Your job job_id ("run_nxs2tiff.sh") has been submitted
Please note that nxs2tiff runs the above job on the DLS cluster (one can monitor its progress in the usual way using qstat). The nxs2tiff command does not return until this job is completed. When the job is completed with success, one should see the following final message:Code Block title Linux output Job job_id exited with exit code 0.
If there are any errors, the message reads as follows:
Code Block title Linux output Job job_id exited with exit code 1.
The errors can be diagnosed with the help of the relevant qsub log files, which can be found in /dls/tmp/tomopy. Please examine your latest run_nxs2tiff.sh.e* style file, found in that directory.
To view the status of all SGE jobs being run by all users, please execute the following qstat command for all users:Anchor A7 A7 Code Block title Linux commands qstat -u "*"
or, equivalently,
Code Block title Linux commands qstat -u \*
To find out which modules have been loaded (added) to the shell environment, please execute:Anchor A8 A8 Code Block title Linux commands module list
As usual, for more information about the module command and its various useful switches and sub-commands, please executeCode Block title Linux commands man module
to display the on-line manual pages on module.