The NEXUS file format

All of DIAD data is packaged as Nexus or .nxs files. The advantage is that each dataset can be packaged with a wealth of metadata. All data is grouped under a master Experiment nexus, that will contain all scans carried out while an experiment is running.

Open

Starting an experiment in GDA

This is what your visit folder will look like:

Open

The folders that are most relevant are:

Experiments - Contains a master nexus for each experiment as defined in GDA [Link]

Nexus - Contains all raw nexus data from each scan.

Processed - All autoprocessed data - image reconstructions and diffraction 2D integrations

Processing - This is the only writable folder for the user. This allows you a space to reprocess or do further processing on data.

Scans data

When you run a scan (imaging or diffraction) the data is saved with a unique scan number within the nexus folder (k11-12345.nxs). The actual raw data is found in the scan subfolder as the detector .H5 file.

Autoprocessing

Both imaging and diffraction data are both autoprocessed on DIAD. Imaging data is reconstructed and diffraction data is reduced from 2D images to 1D patterns.

Imaging data

Raw data is found in nexus/k11-XXXXX/k11-XXXXX-imaging.h5

Depending on the SAVU pipeline, autoprocessed reconstructed data is output as an .h5 in the processed folder within a folder with the scan number and the process list name.

In this case the process list was called full_fd_rr_vo_AST_tiff.

Within this folder the raw data is found in the astra_recon_gpu.h5. If TiffSaver is included in the pipeline then there will also be a folder with each slice saved as a tiff.

When you open the nexus file in DAWN. All Data entries look like this:

This provides a list of entries, most of which are metadata. In the case above this is an imaging dataset and the actual raw data is stored in the entry /imaging/data.

You can often use the dimensions of the data to help you find your data entry. In this case this is an imaging dataset which has the dimensions 2640,1080,1280.

Diffraction data

The autoprocessing pipeline will reduce diffraction data into 1D patterns. The processed data will go into the processed folder and will look like this:

In this case the dataset also produced a folder (labelled _ascii) with .dat files containing the data as XY. The autoprocessing is configured at the start to integrate the 2D image from our detector and to correct geometric aberrations that arise because of our moving beam set up.

If you output .dat files then they will be numbered sequentially. If you do a grid scan they will be numbered left to right from top to bottom whether you select alternating during the scan or not.

The data loaded in dawn will look like this:

The raw data is stored in /entry/diffraction/data and the processed result is stored in /processed/result/data. The processed data is displayed in the central data viewer.

Common metadata locations in nexus