Reconstruction from image data in the HDF format: the tomo-centre and tomo-recon commands
Introduction
The functionality of the Tomo-Recon GUI (in Dawn) relies on a number of scripts which can also be executed directly from the Linux command line. This may be necessary if one aims at reconstructing a non-standard tomography dataset or if one would like to improve the quality of reconstructed slices. For example, it can be useful to execute these scripts from the Linux command line in the following situations:
- flat-Â or dark-field images are stored in separate Nexus datasets or files;
- the information about tomography angles, normally stored in the Nexus scan file, is for some reason missing or inaccurate;
- to optimise ring-artefact suppression (use your own copy of the default settings file, /dls_sw/apps/tomopy/tomopy/src/settings.xml, for this task - see Appendix A);
- to use some more advanced tomography reconstruction options, specified via settings.xml, e.g. to reduce the bit-depth of output images (use your own copy of the default settings file, /dls_sw/apps/tomopy/tomopy/src/settings.xml, for this purpose - see Appendix B);
- to run a batch reconstruction of similar scans (see Appendix C).
The scripts form part of a dedicated module, called tomography. As with any other software-environment module in DLS, please execute:
module load tomography
to make all these scripts available in your current Linux session; the scripts include the following items:
Script Name | Script Type | Script Description | Script Usage | Comment(s) |
---|---|---|---|---|
tomo-centre | Bash | To find an optimal value of the centre of rotation (CoR). | tomo-centre [options] <nexus_file> <output_directory> | Given a slice number and an initial trial value of CoR, this scripts reconstructs this slice with a series of different values of CoR which are smaller or larger than the input value of CoR. These trial reconstructions facilitate the task of finding an optimal value of CoR by visual inspection. The functionality of this script is similar to that of qcentrexml.py for image data in the TIFF format. |
tomo-recon | Bash | To perform tomography reconstruction of data. | tomo-recon [options] <nexus_file> <output_directory> | The functionality of this script is similar to that of recon_arrayxml.py for image data in the TIFF format. |
tomo-fix | Bash | To detect and, if necessary, generate any missing reconstructed images (this is required, for example, if a cluster node is down or in similar circumstances). | tomo-fix [options] <nexus_file> <directory_to_check> | This script is automatically executed whenever tomo-recon is run. |
Please note that absolute paths need to be supplied for the <nexus_file>, <output_directory>, and <directory_to_check> arguments. Please also note that the input <nexus_file> must contain a well-formed tomo_entry, described in more detail on Image data in the HDF format.
Important note
If you copy-and-paste any commands or code from this page, please make sure that all your input arguments are correct before executing anything. In particular, please make sure that the relevant input arguments match the size of your images and that all your input paths to files or directories point to some existing and accessible objects on the file system.
Tip
The easiest way to select (for copying) a long command displayed in a code-block is to double-click anywhere on this command's text.
Log files
Recent log files can be found in a relevant sub-directory of the /dls/tmp/tomopy directory:
~>cd /dls/tmp/tomopy/ ~>ll total 27560 drwxrwxrwx. 2 ssg37927 ssg37927 3313664 Jul 7 16:05 dt64 ... drwxrwxrwx. 2 vxu94780 vxu94780 5787648 Jul 7 17:44 tomo-centre drwxrwxrwx. 2 vxu94780 vxu94780 118784 Jul 4 21:13 tomo-compress drwxrwxrwx. 2 fsy24678 fsy24678 18866176 Jul 7 13:08 tomo-recon ~>
tomo-centre
The tomo-centre script expects a certain number of arguments and provides some additional options to choose from. When executed, it first creates an appropriate Linux environment and then invokes a Python script called selection_recon.py. All the input arguments and options, supplied for running tomo-centre, are passed in to selection_recon.py. As usual, a more detailed description of all those arguments and options can be viewed by executing:
tomo-centre --help
or
tomo-centre -h
At the time of writing this section (18 Sep 2017), either of the above two commands outputs the following description:
~>tomo-centre -h Welcome to the DLS compute cluster For MPI jobs, please use 'module load openmpi'. If using a different OpenMPI installation, or manually specifying path to OpenMPI, option '-mca orte_forward_job_control 1' must be added to mpirun to ensure cluster functionality. Please report any issues to linux.manager@diamond.ac.uk Loading 64-bit Oracle instantclient, version 11.2 Loading 64-bit python, version 2.7.2 Loading 64-bit numpy, version 1.6.1 Usage: selection_recon.py [options] data output_directory Options: -h, --help show this help message and exit -m MACHINES, --machines=MACHINES Number of machines to deploy to -s SLICE, --slice=SLICE Slice selected for processing -t TEMPLATE, --template=TEMPLATE Template XML file -w WSAMP, --width_sample=WSAMP Set the subsampling of the sinograms width -l LSAMP, --length_sample=LSAMP Set the subsampling of the sinograms length -c CSTART, --cstart=CSTART Starting value for the centre of rotation --ctot=CTOT Total number of different values for the centre of rotation --cstep=CSTEP The step between two consecutive values for the centre of rotation -r RUN_SLICES, --run_slices_loc=RUN_SLICES set the run_slices.sh location -n, --new_cluster use the new cluster --dark_file=DARK_FILE Path to the file containing dark images --dark_path=DARK_PATH path in the dark file to the data --flat_file=FLAT_FILE Path to the file containing flat images --flat_path=FLAT_PATH path in the dark file to the data --recon_range=RECON_RANGE range for the reconstruction to be done over --d0f1 If option included (True), use 0's for dark- and 1's for flat-field images --scan_id If option included (True), incorporate the ID of the input Nexus scan file into output filenames ~>
Option Switch | Option's Default Value | Comment(s) |
---|---|---|
-m | 2 | |
-s | None (Python keyword) | Slice number must be less than the vertical size (in pixels) of the images. |
-t | /dls_sw/apps/tomopy/tomopy/src/settings.xml | 1. Absolute path needs to be supplied (if different from default). |
-w | 1 | |
-l | 1 | |
-c | 2004 | |
-r | /dls_sw/apps/tomopy/tomopy/bin/run_slices.sh | Path to a tomography reconstruction script to be used (advanced users only). |
-n | False | |
--dark_file | n/a | 1. Path to the Nexus file containing dark-field images. |
--dark_path | /entry1/pco1_hw_hdf/data | 1. To be supplied only if --dark_file option is used (see --dark_file). |
--flat_file | n/a | 1. Path to the Nexus file containing flat-field images. |
--flat_path | /entry1/pco1_hw_hdf/data | 1. To be supplied only if --flat_file option is used (see --flat_file). |
--recon_range | -1 | If default is used, angular range is automatically determined from rotation-angle data stored in the input Nexus file, ie <nexus_file>. |
--d0f1 | False (implicit) | If this option is included on the command line, then a synthetic dark-field image consisting of all 0's and another synthetic flat-field image of all 1's are used during |
--scan_id | False (implicit) | If this option is included on the command line, then the scan ID is automatically included in the output filenames, eg for Nexus scan file 91809.nxs, a typical output filename |
BASIC EXAMPLE
In most cases, basic use of tomo-centre is adequate.
tomo-centre -s 1000 -c 1269.5 --ctot 10 --cstep 0.1 -n /dls/i13/data/2014/mt9377-2/raw/29384.nxs /dls/i13/data/2014/mt9377-2/processing/reconstruction/29384/
In the above basic example, the command structure is as follows:
tomo-centre [options] <nexus_file> <output_directory>
Command Argument(s) | Example Command Argument(s) | Comment(s) |
---|---|---|
[options] | -s 1000 | SLICE (used by the s-option) should not exceed the (pixel) height of raw images. |
<nexus_file> | /dls/i13/data/2014/mt9377-2/raw/29384.nxs | |
<output_directory> | /dls/i13/data/2014/mt9377-2/processing/reconstruction/29384/ | If this output directory doesn't already exist, it will automatically be created. |
MOST EXTENSIVE EXAMPLE
In more complicated cases, tomo-centre needs to be executed with  all or an appropriate selection of additional input arguments.
tomo-centre -s 1000 -c 1269.5 --ctot 10 --cstep 0.1 -n --dark_file=/dls/i13/data/2014/mt9377-2/raw/29375.nxs --dark_path=/entry1/pco1_sw/data --flat_file=/dls/i13/data/2014/mt9377-2/raw/29383.nxs --flat_path=/entry1/pco1_sw/data /dls/i13/data/2014/mt9377-2/raw/29384.nxs /dls/i13/data/2014/mt9377-2/processing/reconstruction/29384/ --recon_range=180.0 --template=/dls/i13/data/2014/mt9377-2/processing/drain1/settingsHDF.xml
In the above most extensive example, the command structure is as follows:
tomo-centre [options] <nexus_file> <output_directory>
Command Argument(s) | Example Command Argument(s) | Comment(s) |
---|---|---|
[options] | -s 1000 | SLICE (used by the s-option) should not exceed the (pixel) height of raw images. |
<nexus_file> | /dls/i13/data/2014/mt9377-2/raw/29384.nxs | |
<output_directory> | /dls/i13/data/2014/mt9377-2/processing/reconstruction/29384/ | If this output directory doesn't already exist, it will automatically be created. |
tomo-recon
The tomo-recon script expects a certain number of arguments and provides some additional options to choose from. When executed, it first creates an appropriate Linux environment and then invokes a Python script called full_recon.py. All the input arguments and options, supplied for running tomo-recon, are passed in to full_recon.py. As usual, a more detailed description of all those arguments and options can be viewed by executing:
tomo-recon --help
or
tomo-recon -h
At the time of writing this section (18 Sep 2017), either of the above two commands outputs the following description:
~>tomo-recon -h Welcome to the DLS compute cluster For MPI jobs, please use 'module load openmpi'. If using a different OpenMPI installation, or manually specifying path to OpenMPI, option '-mca orte_forward_job_control 1' must be added to mpirun to ensure cluster functionality. Please report any issues to linux.manager@diamond.ac.uk Loading 64-bit Oracle instantclient, version 11.2 Loading 64-bit python, version 2.7.2 Loading 64-bit numpy, version 1.6.1 Starting Full Recon Usage: full_recon.py [options] data output_directory Options: -h, --help show this help message and exit -m MACHINES, --machines=MACHINES Number of machines to deploy to -b SLICE_BEGIN, --slice_begin=SLICE_BEGIN Start Slice number -e SLICE_END, --slice_end=SLICE_END End Slice Number -t TEMPLATE, --template=TEMPLATE Template XML file -w WSAMP, --width_sample=WSAMP Set the subsampling of the sinograms width -l LSAMP, --length_sample=LSAMP Set the subsampling of the sinograms length -c CENTRE, --centre=CENTRE Set the centre of rotation -r RUN_SLICES, --run_slices_loc=RUN_SLICES set the run_slices.sh location -n, --new_cluster use the new cluster -p, --preview Run a preview reconstruction -a, --angles Use angular information to reconstruct, do not use with a ROI -o, --old_cluster Use the old cluster --dark_file=DARK_FILE Path to the file containing dark images --dark_path=DARK_PATH path in the dark file to the data --flat_file=FLAT_FILE Path to the file containing flat images --flat_path=FLAT_PATH path in the dark file to the data --recon_range=RECON_RANGE range for the reconstruction to be done over --d0f1 If option included (True), use 0's for dark- and 1's for flat-field images --scan_id If option included (True), incorporate the ID of the input Nexus scan file into output filenames ~>
Option Switch | Option's Default Value | Comment(s) |
---|---|---|
-m | 2 | |
-b | 0 | |
-e | 128 | |
-t | /dls_sw/apps/tomopy/tomopy/src/settings.xml | 1. Absolute path needs to be supplied (if different from default). |
-w | 1 | |
-l | 1 | |
-r | /dls_sw/apps/tomopy/tomopy/bin/run_slices.sh | Path to a tomography reconstruction script to be used (advanced users). |
-n | False | |
-p | False | |
-a | False | |
-o | False | |
--dark_file | None (Python keyword) | 1. Path to the Nexus file containing dark-field images. |
--dark_path | /entry1/pco1_hw_hdf/data | 1. To be supplied only if --dark_file option is used (see --dark_file). |
--flat_file | None (Python keyword) | 1. Path to the Nexus file containing flat-field images. |
--flat_path | /entry1/pco1_hw_hdf/data | 1. To be supplied only if --flat_file option is used (see --flat_file). |
--recon_range | -1 | If default is used, angular range is automatically determined from rotation-angle data stored in the input Nexus file, ie <nexus_file>. This option is particularly useful for limited-angle reconstruction. |
--d0f1 | False (implicit) | If this option is included on the command line, then a synthetic dark-field image consisting of all 0's and another synthetic flat-field image of all 1's are used during |
--scan_id | False (implicit) | If this option is included on the command line, then the scan ID is automatically included in the output filenames, eg for Nexus scan file 91809.nxs, a typical output filename |
BASIC EXAMPLE
In most cases, basic use of tomo-recon is adequate.
tomo-recon -m 20 -b 0 -e 2159 -c 1271.1 /dls/i13/data/2014/mt9377-2/raw/30119.nxs /dls/i13/data/2014/mt9377-2/processing/reconstruction/30119/
In the above basic example, the command structure is as follows:
tomo-recon [options] <nexus_file> <output_directory>
Command Argument(s) | Example Command Argument(s) | Comment(s) |
---|---|---|
[options] | -m 20 | SLICE_END (used by the e-option) should not exceed the (pixel) height of raw images. |
<nexus_file> | /dls/i13/data/2014/mt9377-2/raw/30119.nxs | |
<output_directory> | /dls/i13/data/2014/mt9377-2/processing/reconstruction/30119/ | This output directory must exist before executing tomo-recon. |
MOST EXTENSIVE EXAMPLE
In more complicated cases,  tomo-recon needs to be executed with  all or an appropriate selection of additional input arguments.
tomo-recon -m 20 -b 0 -e 2159 -c 1271.1 --dark_file=/dls/i13/data/2014/mt9377-2/raw/29375.nxs --dark_path=/entry1/pco1_sw/data --flat_file=/dls/i13/data/2014/mt9377-2/raw/30535.nxs --flat_path=/entry1/pco1_sw/data --recon_range=180.0 --template=/dls/i13/data/2014/mt9377-2/processing/drain1/settingsHDF.xml /dls/i13/data/2014/mt9377-2/raw/30119.nxs /dls/i13/data/2014/mt9377-2/processing/reconstruction/30119/
In the above most extensive example, the command structure is as follows:
tomo-recon [options] <nexus_file> <output_directory>
Command Argument(s) | Example Command Argument(s) | Comment(s) |
---|---|---|
[options] | -m 20 | SLICE_END (used by the e-option) should not exceed the (pixel) height of raw images. |
<nexus_file> | /dls/i13/data/2014/mt9377-2/raw/30119.nxs | |
<output_directory> | /dls/i13/data/2014/mt9377-2/processing/reconstruction/30119/ | This output directory must exist before executing tomo-recon. |
tomo-fix
The tomo-fix script expects a certain number of arguments and provides some additional options to choose from. When executed, it first creates an appropriate Linux environment and then invokes a Python script called tomo-fix.py. All the input arguments and options, supplied for running tomo-fix, are passed in to tomo-fix.py. As mentioned at the top of this page, this script is automatically executed whenever tomo-recon is run. As usual, a more detailed description of all those arguments and options can be viewed by executing:
tomo-fix --help
or
tomo-fix -h
At the time of writing this section (18 Sep 2017), either of the above two commands outputs the following description:
~>tomo-fix -h Loading 64-bit Oracle instantclient, version 11.2 Loading 64-bit python, version 2.7.2 Loading 64-bit numpy, version 1.6.1 Usage: tomo-fix.py [options] nexus_file directory_to_check Options: --version show program's version number and exit -h, --help show this help message and exit -m MACHINES, --machines=MACHINES Number of machines to deploy to -b SLICE_BEGIN, --slice_begin=SLICE_BEGIN Start Slice number -e SLICE_END, --slice_end=SLICE_END End Slice Number -t TEMPLATE, --template=TEMPLATE Template XML file -w WSAMP, --width_sample=WSAMP Set the subsampling of the sinograms width -l LSAMP, --length_sample=LSAMP Set the subsampling of the sinograms length -c CENTRE, --centre=CENTRE Set the centre of rotation -r RUN_SLICES, --run_slices_loc=RUN_SLICES set the run_slices.sh location -n, --new_cluster use the new cluster -p, --preview Run a preview reconstruction -a, --angles Use angular information to reconstruct, do not use with a ROI -o, --old_cluster Use the old cluster --dark_file=DARK_FILE Path to the file containing dark images --dark_path=DARK_PATH path in the dark file to the data --flat_file=FLAT_FILE Path to the file containing flat images --flat_path=FLAT_PATH path in the dark file to the data --recon_range=RECON_RANGE range for the reconstruction to be done over --d0f1 If option included (True), use 0's for dark- and 1's for flat-field images --scan_id If option included (True), incorporate the ID of the input Nexus scan file into output filenames ~>
Option Switch | Option's Default Value | Comment(s) |
---|---|---|
-m | 2 | |
-b | 0 | |
-e | 128 | |
-t | /dls_sw/apps/tomopy/tomopy/src/settings.xml | 1. Absolute path needs to be supplied (if different from default). |
-w | 1 | |
-l | 1 | |
-r | /dls_sw/apps/tomopy/tomopy/bin/run_slices.sh | Path to a tomography reconstruction script to be used (advanced users). |
-n | False | |
-p | False | |
-a | False | |
-o | False | |
--dark_file | n/a | 1. Path to the Nexus file containing dark-field images. |
--dark_path | n/a | 1.To be supplied only if --dark_file option is used (see --dark_file). |
--flat_file | n/a | 1. Path to the Nexus file containing flat-field images. |
--flat_path | n/a | 1. To be supplied only if --flat_file option is used (see --flat_file). |
--d0f1 | False (implicit) | If this option is included on the command line, then a synthetic dark-field image consisting of all 0's and another synthetic flat-field image of all 1's are used during |
--scan_id | False (implicit) | If this option is included on the command line, then the scan ID is automatically included in the output filenames, eg for Nexus scan file 91809.nxs, a typical output filename |
BASIC EXAMPLE
In most cases, basic use of tomo-fix is adequate.
tomo-fix -m 20 -b 0 -e 2159 -c 1271.1 /dls/i13/data/2014/mt9377-2/raw/30119.nxs /dls/i13/data/2014/mt9377-2/processing/reconstruction/30119/
In the above basic example, the command structure is as follows:
tomo-fix [options] <nexus_file> <directory_to_check>
Command Argument(s) | Example Command Argument(s) | Comment(s) |
---|---|---|
[options] | -m 20 | SLICE_END (used by the e-option) should not exceed the (pixel) height of raw images. |
<nexus_file> | /dls/i13/data/2014/mt9377-2/raw/30119.nxs | |
<directory_to_check> | /dls/i13/data/2014/mt9377-2/processing/reconstruction/30119/ | This output directory must exist before executing tomo-recon. |
MOST EXTENSIVE EXAMPLE
In more complicated cases, tomo-fix needs to be executed with  all or an appropriate selection of additional input arguments.
tomo-fix -m 20 -b 0 -e 2159 -c 1271.1 --dark_file=/dls/i13/data/2014/mt9377-2/raw/29375.nxs --dark_path=/entry1/pco1_sw/data --flat_file=/dls/i13/data/2014/mt9377-2/raw/30535.nxs --flat_path=/entry1/pco1_sw/data --recon_range=180.0 --template=/dls/i13/data/2014/mt9377-2/processing/drain1/settingsHDF.xml /dls/i13/data/2014/mt9377-2/raw/30119.nxs /dls/i13/data/2014/mt9377-2/processing/reconstruction/30119/
In the above most extensive example, the command structure is as follows:
tomo-fix [options] <nexus_file> <directory_to_check>
Command Argument(s) | Example Command Argument(s) | Comment(s) |
---|---|---|
[options] | -m 20 | SLICE_END (used by the e-option) should not exceed the (pixel) height of raw images. |
<nexus_file> | /dls/i13/data/2014/mt9377-2/raw/30119.nxs | |
<directory_to_check> | /dls/i13/data/2014/mt9377-2/processing/reconstruction/30119/ | This output directory must exist before executing tomo-recon. |
Appendix A: Ring-artefact suppression
Ring-artefact suppression can be performed as part of the reconstruction process with the help of an appropriately configured settings.xml. Note that the default /dls_sw/apps/tomopy/tomopy/src/settings.xml has ring-artefact suppression enabled with the following parameters:Â
... <RingArtefacts> <Type info="No, Column, AML">AML</Type> <ParameterN>0.000</ParameterN> <ParameterR>0.0050</ParameterR> <NumSeries info="1 - default, if greater, then nonlinear">1.0</NumSeries> </RingArtefacts> ...
To optimise ring-artefact suppression, you need to first modify the ParameterR or NumSeries parameters in your own copy of the default /dls_sw/apps/tomopy/tomopy/src/settings.xml and then explicitly specify a path to this modified copy to run tomo-recon (use the t-option) in order to reconstruct a few representative slices for subsequent visual inspection (this workflow is similar to that employed for finding an optimal value of CoR). ParameterR controls the strength of ring-artefact suppression, and it is best to change it in order-of-magnitude steps, e. g. 0.5, 0.05, 0.005 (default), 0.0005. Note that the smaller ParameterR, the more aggressive ring-artefact suppression. NumSeries is used for controlling the high-aspect-ratio compensation (for plate-like samples).Â
If desired, ring-suppression can be completely switched off by changing Type (from the default value of AML) to No:
... <RingArtefacts> <Type info="No, Column, AML">No</Type> <ParameterN>0.000</ParameterN> <ParameterR>0.0050</ParameterR> <NumSeries info="1 - default, if greater, then nonlinear">1.0</NumSeries> </RingArtefacts> ...
Appendix B: Optional bit-depth reduction of output TIFFs
Bit-depth reduction can be performed as part of the reconstruction process with the help of an appropriately configured settings.xml:
... <OutputData> ... <BitsType info="Input, Given">Given</BitsType> <Bits>32</Bits> <Restrictions info="Yes, No">No</Restrictions> <ValueMin>0.5</ValueMin> <ValueMax>1.1</ValueMax> ... </OutputData> ...
The above snippet of XML shows the default (32-bit) settings (in the default /dls_sw/apps/tomopy/tomopy/src/settings.xml file). Before modifying these settings (in your own copy of the default /dls_sw/apps/tomopy/tomopy/src/settings.xml), it is crucial to find some sensible replacement values for the above ValueMin and ValueMax. Since these values are image-dependent, it is best to select them with the help of a histogram (e.g. in ImageJ) of your typical, 32-bit reconstructed slice. Â
Alternatively, bit-depth reduction can be done after the reconstruction process, as described in Extraction of TIFF images from image data in the HDF format (with optional bit-depth reduction) and related matters
Appendix C: Batch reconstruction
It is sometimes desirable to reconstruct a batch of similar tomography scans. The Bash script presented below provides a simple example of how to do it for a sequence of consecutive scans sharing the same centre of rotation. More precisely, this script reconstructs a batch of 4 consecutive scans, numbered from 10000 to 10003, using the same centre of rotation, 1234.5, and the same flat- and dark-field datasets, stored in 2 separate Nexus files (one for flats and the other for darks). The values of 0 and 2159 for the b and, respectively, the e option have been chosen to match the full size of images produced by PCO Edge.
Please note that, to prevent the DLS compute cluster from getting overloaded, the script reconstructs scans one after another (sequentially) as opposed to reconstructing all scans in parallel at the same time. Â
#! /bin/bash echo "Running batch reconstruction..." ### USER INPUT: START # What is your visit's ID and year? visitID="xx12345-6" year=2016 # What is the COMMON centre of rotation for all the scans? CENTRE=1234.5 # What is the first scan number? START=10000 # What is the last scan number? END=10003 # Which directory contains the scans (with sample projections)? RAWDIR="/dls/i13/data/${year}/${visitID}/raw" # Which directory contains the flat scan? FLATDIR="/dls/i13/data/${year}/${visitID}/raw" # What is the flat-scan number? FLAT=11111 # Which directory contains the dark scan? DARKDIR="/dls/i13/data/${year}/${visitID}/raw" # What is the dark-scan number? DARK=66666 # Which directory is to be used for saving reconstructed images in scan-numbered sub-directories? PRODIR="/dls/i13/data/${year}/${visitID}/processing/reconstruction" ### USER INPUT: END module add tomography echo "BATCH START = ${START}" echo "BATCH END = ${END}" for (( i=$START; i<=$END; i++ )) do cd $PRODIR echo "in directory: "`pwd`  echo "making reconstruction directory for scan: $i" [ -d $i ] || mkdir $i echo "reconstructing scan: $i" echo "tomo-recon command is:" # the next two lines print out the command before it is executed (in the 3rd line), which is useful for testing, etc  echo \ tomo-recon -m 20 -b 0 -e 2159 -c $CENTRE --dark_file=$DARKDIR/$DARK.nxs --dark_path=/entry1/pco1_hw_hdf_nochunking/data --flat_file=$FLATDIR/$FLAT.nxs --flat_path=/entry1/pco1_hw_hdf_nochunking/data $RAWDIR/$i.nxs $PRODIR/$i/ --recon_range=180.0 --template=/dls/i13/data/2014/visitID/processing/settingsHDF.xml tomo-recon -m 20 -b 0 -e 2159 -c $CENTRE --dark_file=$DARKDIR/$DARK.nxs --dark_path=/entry1/pco1_hw_hdf_nochunking/data --flat_file=$FLATDIR/$FLAT.nxs --flat_path=/entry1/pco1_hw_hdf_nochunking/data $RAWDIR/$i.nxs $PRODIR/$i/ --recon_range=180.0 --template=/dls/i13/data/2014/visitID/processing/settingsHDF.xml echo "finished reconstructing scan: $i" done echo "Finished batch reconstruction."
If your batch contains scans which are not consecutively numbered or if they do not share the common centre of rotation, then the script presented below can be used to reconstruct such a batch. More precisely, this script reconstructs a batch of 2 non-consecutive scans, numbered 10000 and 10003, using 2 different centres of rotation (1234.5 and 5432.1, respectively), and the same flat- and dark-field datasets, stored in 2 separate Nexus files (one for flats and the other for darks):
#! /bin/bash echo "Running batch reconstruction..." # Declare associative array declare -A arr ### USER INPUT: START # What is your visit's ID and year? visitID="xx12345-6" year=2016 # What are your scan numbers and the corresponding centres of rotations? # TEMPLATE: arr[scan_number]=centre_of_rotation arr[10000]=1234.5 arr[10003]=5432.1 # Which directory contains the scans (with sample projections)? RAWDIR="/dls/i13/data/${year}/${visitID}/raw" # Which directory contains the flat scan? FLATDIR="/dls/i13/data/${year}/${visitID}/raw" # What is the flat-scan number? FLAT=11111 # Which directory contains the dark scan? DARKDIR="/dls/i13/data/${year}/${visitID}/raw" # What is the dark-scan number? DARK=66666 # Which directory is to be used for saving reconstructed images in scan-numbered sub-directories? PRODIR="/dls/i13/data/${year}/${visitID}/processing/reconstruction" ### USER INPUT: END module add tomography echo "BATCH:" item=1 for i in ${!arr[@]]} do echo ${item}. scan=${i} CoR=${arr[${i}]} item=$((item+1)) done for i in ${!arr[@]]} do cd $PRODIR echo "in directory: "`pwd` echo "making reconstruction directory for scan: $i" [ -d $i ] || mkdir $i echo "reconstructing scan: $i" echo "tomo-recon command is:" # the next two lines print out the command before it is executed (in the 3rd line), which is useful for testing, etc echo \ tomo-recon -m 20 -b 0 -e 2159 -c ${arr[${i}]} --dark_file=$DARKDIR/$DARK.nxs --dark_path=/entry1/pco1_hw_hdf_nochunking/data --flat_file=$FLATDIR/$FLAT.nxs --flat_path=/entry1/pco1_hw_hdf_nochunking/data $RAWDIR/$i.nxs $PRODIR/$i/ --recon_range=180.0 --template=/dls/i13/data/2014/visitID/processing/settingsHDF.xml tomo-recon -m 20 -b 0 -e 2159 -c ${arr[${i}]} --dark_file=$DARKDIR/$DARK.nxs --dark_path=/entry1/pco1_hw_hdf_nochunking/data --flat_file=$FLATDIR/$FLAT.nxs --flat_path=/entry1/pco1_hw_hdf_nochunking/data $RAWDIR/$i.nxs $PRODIR/$i/ --recon_range=180.0 --template=/dls/i13/data/2014/visitID/processing/settingsHDF.xml echo "finished reconstructing scan: $i" done # Clear the array unset arr echo "Finished batch reconstruction."
To use either of these two example scripts for batch processing, please follow these steps:
- copy-and-paste the above code into your own .sh file (e.g. mybatch.sh);
- modify the script in this file to suit your particular needs, making sure that all input arguments match the size of your images and that all input paths to files or directories point to some existing and accessible objects (the script automatically creates the output directory, if it does not exist); Â
- save the file;
- make the file executable (e.g. chmod u+x mybatch.sh);
- execute the script (e.g. ./mybatch.sh)
It is always a good practice to first test your new script without running any jobs on the compute cluster. This can be accomplished by commenting out (c.f. the leading #) the last occurrence of the two identical lines beginning with 'tomo-recon', i.e. Â Â Â Â
echo \ tomo-recon -m 20 -b 0 -e 2159 -c $CENTRE --dark_file=$DARKDIR/$DARK.nxs --dark_path=/entry1/pco1_hw_hdf_nochunking/data --flat_file=$FLATDIR/$FLAT.nxs --flat_path=/entry1/pco1_hw_hdf_nochunking/data $RAWDIR/$i.nxs $PRODIR/$i/ --recon_range=180.0 --template=/dls/i13/data/2014/visitID/processing/settingsHDF.xml #tomo-recon -m 20 -b 0 -e 2159 -c $CENTRE --dark_file=$DARKDIR/$DARK.nxs --dark_path=/entry1/pco1_hw_hdf_nochunking/data --flat_file=$FLATDIR/$FLAT.nxs --flat_path=/entry1/pco1_hw_hdf_nochunking/data $RAWDIR/$i.nxs $PRODIR/$i/ --recon_range=180.0 --template=/dls/i13/data/2014/visitID/processing/settingsHDF.xml
Please note that the first occurrence of this line simply prints out the intended tomo-recon command with its expanded arguments (which is useful for spotting any typos, etc), whereas the second occurrence would normally execute this command for real, sending a number of jobs to run on the compute cluster. This dry run should help one verify that all input arguments are correct and that all input paths to files or directories point to some existing and accessible objects on the file system (the script automatically creates the output directory, if the latter does not already exist). If no problems are detected with the script, then the commented-out line needs, of course, to be commented back in before the script can be executed for real. Â
Appendix D: Troubleshooting
If the reconstruction scripts do not produce any output files, experienced users can find the logs at the following location:
/dls/tmp/tomopy/tomo-centre/ /dls/tmp/tomopy/tomo-recon/
For finding the latest log files, sort the contents and open the logfile for example with gedit:
[user@machine tomo-centre]$ ls -lthr -rw-r--r--. 1 fedid123 fedid123 0 Feb 13 15:43 run_slices.sh.po22535829 -rw-r--r--. 1 fedid123 fedid123 0 Feb 13 15:43 run_slices.sh.pe22535829 -rw-r--r--. 1 fedid123 fedid123 599 Feb 13 15:43 run_slices.sh.e22535827 -rw-r--r--. 1 fedid123 fedid123 599 Feb 13 15:43 run_slices.sh.e22535828 -rw-r--r--. 1 fedid123 fedid123 599 Feb 13 15:43 run_slices.sh.e22535829 -rw-r--r--. 1 fedid123 fedid123 599 Feb 13 15:43 run_slices.sh.e22535826 -rw-r--r--. 1 fedid123 fedid123 4.1K Feb 13 15:43 run_slices.sh.o22535829 -rw-r--r--. 1 fedid123 fedid123 4.1K Feb 13 15:43 run_slices.sh.o22535828 -rw-r--r--. 1 fedid123 fedid123 4.1K Feb 13 15:43 run_slices.sh.o22535826 -rw-r--r--. 1 fedid123 fedid123 4.1K Feb 13 15:43 run_slices.sh.o22535827 [user@machine tomo-centre] gedit run_slices.sh.o22535827