Release Notes

Owned by Jacob Filik
Last updated: Apr 02, 2019 by Peter Chang
7 min read

DAWN 2.13

New features - 

  • General improvements
    • Welcome Screen removed.
    • Trace Perspective removed.
    • DataVis, Mapping, Processing and Powder calibration have a common file menu
      • Includes recently loaded files, recent folders history and importing images as a stack
      • Add tooltips for table entries so that long strings can be read without resizing GUI
    • Plotting improvements:
      • Make axis inversions and image origin mutually consistent
      • Per plot preference - use the pull-down menu (small white triangle on rightmost of plot toolbar - could be wrapped to a second line if the plot is narrowly sized), select the X/Y submenu then click on the Plot Preference item which 3rd from the bottom to bring up the dialog.
    • Fast mask tool as option to mask a line with a set thickness, and also to automatically mask on release of a region
    • Option to change optimiser added to function fitting tool
    • Text shown next to the perspective icon in the main toolbar by default
    • Improve thread use in XAFS tool
  • DataVis perspective improvements
    • Quick average, sum and taking views of datasets
      • Save these to Nexus Files
    • Improve sorting of files against labels
    • Metadata Dialog added for viewing file metadata
    • Slicing and plot type no longer resets if configured before the dataset is selected for display
    • Changed toolpage stack processing to user Processing code in DataVis
    • improve error messages in status bar
    • Click to open from the Project Explorer now works properly
    • Folder preview now works for all files (.dat/cbf/edf...) files and NeXus/HDF5
  • Mapping perspective improvements
    • Option to view multi channel detectors as an image, the sum, or individual channels
  • Processing
    • default Nexus tags added to the processed data files
    • Processing step to load Nexus detector transformations from scanned detectors
    • Internal python engine added
      • Larch processing steps added for XAFS
      • PyMca's SNIP background removal for XRF
    • hide some model fields by default
    • show successes/failures in operation logging by colour
    • allow plots to be customized
  • RIXS improvements
    • Add range selection to QuickRIXS
    • Add counts per photon field to models
    • Save fits in image reduction
    • Make preliminary version of combined image reduction
    • Add energy range to model to specify where correlation happens
    • Add smoothed 2D dark image support to remove backgrounds where there is lateral variation
    • Add normalization dataset path to model to allow choice of which dataset to use for normalizing spectrum
    • Permit negative pixel values in background subtracted images to avoid summation bias
  • ScisoftPy improvements
    • Add array_like support to various function
    • Expose multiple axes for reduction methods
    • Add keepdims to reduction functions and methods

Bug Fixes -

Many bug fixes went into the release, please refer to JIRA for issues you are particularly interested in.

Removed features -

The Welcome Screen and Trace perspective, along with several other unused features, to keep the code base tidy.


DAWN 2.12

Internal branch for testing.

DAWN 2.11

New features - 

  • General improvements
    • Python plotting updated to use new 3D plotting
    • New "Transfer" menu to make opening data in multiple perspectives easier
    • Speed up loading HDF5 files with huge number of chunks
    • Add new number entry widget for masking tool
  • DataVis perspective improvements
    • Plotting status bar added, data slicing more responsive
  • Mapping perspective improvements
    • Make file loading lazier
    • Improved histogramming behaviour
  • Processing
    • Processed data now writes to /processed/result as apposed to /entry/result (keeps linked raw data in /entry)
    • Add ability to automatically load processed results in DataVis
    • Processing setup dialog improved, more responsive
    • Processing step to average frames from a grid scan using a mask to reject frame
  • RIXS improvements
    • Add linear transformation to QuickRIXS
    • Add new align dialog to PostRIXS that allows for manual adjustments
    • Improve FWHM minimization algorithm for finding slope of elastic lines
    • Add slope overrride to elastic line fitting


Bug Fixes -

Many bug fixes went into the release, please refer to JIRA for issues you are particularly interested in.

Removed features -

The old 3D plotting for surfaces, lines, isosurfaces and volumes has been removed from this release, along with several other unused features, to keep the code base tidy.

DAWN 2.10

New features - 

  • General improvements
    • Upgrade of the User Interface framework - better support for HiDPI displays
    • PyDev updated to 6.4.4
  • DataVis perspective improvements
    • Improvements to 3D plotting efficiency
    • beta version - Hyper4D visualization mode - dynamic viewing of grid scans of imaging detectors
    • beta version - Volume with orthogonal slices visualisation mode
    • beta version - XMCD dialog
  • Spectroscopy
    • New operations LarchXAFSRelativeNormalisation and LarchXAFSNormalisation
    • DataReduction2DTool: add axis export options, increase performance
  • RIXS improvements
    • add alignment tool to PostRIXS with resampling and averaging option
    • allow single values to be plotting in PostRIXS
    • add window size option for cut-off pixels
    • improve dark image profile fitting
    • fix inconsistent offset in spectra
    • save individual spectra
    • widen elastic peak fitting zone
    • add cosmic ray remover option for dark image profile

DAWN 2.9

New features –

  • General improvements
    • Export 1D plots can now write columns or rows of data
    • ScisoftPy now supports Python3 (note, for Python2, the six module needs to be at least version 1.6.0)
    • Add support for reading XDI files
    • Data Reduction Tool: automatically load reduced files in DataVis, allow deselecting regions, == 
  • DataVis perspective improvements:
    • Use default attributes in NXS files to determine which dataset should be loaded automatically
    • New volume rendering visualisation mode
    • Improvements to surface and waterfall plot modes (faster, supports larger data sizes)
  • New tools:
    • EDE Calibration Tool: access via menubar Tools
  • Processing improvements:
    • New operations using Larch as backend. These will keep a python interpreter running in the background for optimal performance:
      • LarchXAFSPostEdgeBackground: normalize and subtract post edge background using Larch's AUTOBK algorithm
      • LarchXAFSFFT: forward Fourier transform
      • LarchXAFSRFT: reverse Fourier transform
    • Fix live plot view for Python script based operations
    • Draw OperationDataForDisplay datasets in live plot view
    • For RIXS image reduction:
      • Handle dark image background in SubtractFittedBackground
      • Fix slope artefact in spectrum for RIXS images
      • Allow frames to be skipped when reducing images
  • New PostRIXS perspective allows quick inspection of processed data (nightly has more features)

Bug Fixes -

Many bug fixes went into the release, please refer to JIRA for issues you are particularly interested in.

DAWN 2.8

New features –

  • Improvements to DataVis perspective (http://confluence.diamond.ac.uk/display/DT/Quick+Introduction+to+DataVis)
    • New data visualization modes - Text table spreadsheet view and new 3d plots, including waterfall plots
    • Use a dataset from the file as a label (scan command, sample name, temperature...) in the Data Files table, or a user editable value, sort files by the label, or file name (order of plotted lines reflects order in table).
    • Option to filter datasets in the Datasets view so only signal datasets in NXdata groups are visible 
    • More information shown in Nexus Tree Dialog (dataset chunking, max size etc...)
    • Slicing can now be set using axis values rather then dataset indices
    • New methods for averaging and subtracting XY datasets
    • Access Convert options by right-clicking on files in Data Files table
    • New Join files method to stack data from multiple files together
  • New plot tools/ tool features
    • Box Profile Tool - Drop-down menu to select which X/Y profiles to plot
    • Data Reduction Tool - take image composed of spectra/diffraction patterns in X and scanned variable (temperature/time...) in Y and compare or average the spectra/patterns.
    • Monitor Tool - now has oscilloscope or running average mode
  • Processing improvements
    • Auxiliary plots and logging text display (used in some new operations)
    • Extract Ring Intensities - Saves q, azimuthal angle and intensity of every pixel in a specified q range to file. Used to fit for strain in external scripts
    • USAXS Processing Chain - For the correction of Bonse-Harte USAXS results for subsequent analysis
    • Remapping detector frames into q space
    • Solid angle correction - Now takes into account detector distance and broken out as separate plugin
    • Self-absorption correction for SAXS
    • Background subtraction improvements
      • Subtraction of datasets on a calibrated scale (i.e. motor position) with interpolation where needed
      • Subtraction of datasets from a nominated internal filepath
      • Subtraction of 1D datasets remapped into 2D from 2D images
      • Autoscaling subtraction - reducing two images to I vs. q plots, matching their intensities at a particular q-range, deducing the scaling factor and then subtracting the scaled version of frame B from A
    • Integration routines - Now will produce NaNs instead of zeros when masked pixels are integrated
    • Integration routines - q list has been reformatted and q can now be in inverse nanometers as well as meters
    • Remove NaNs from dataset - when combined with integration routine changes, will clean up masked data from a dataset preventing the writing of redundant datapoints
    • Calculation of errors in q - First pass at most significant error sources in q (as opposed to intensity)
    • Azimuthal Integration Difference Image - Added functionality to pick between subtraction or division of frames, the latter being useful for flatfield verification.
    • New operations: MinimumFramesOperation and MaximumFramesOperation
    • Savu plugin operations page: refactor, improve stability and fix GUI
    • The Powder Lines Tool now supports multiple files and adjustment of the zero-pressure unit cell volume.
    • The XPDF Processing allows scaling of the Compton scattering subtracted from the data, as well as adjustment of the polarization of the incident beam.
    • Subtract fitted background - Subtract a background level from image whose noisy background can be described by a Gaussian distribution. It also allows a dark image to be used to estimate a vertically varying profile.
    • RIXS elastic line reduction - Find the tilt in the detector from the measurement of a sample that provides an "elastic" line in the spectrometer image. It also calculates the energy dispersion value when an energy scan's data is given to calibrate the spectrometer resolution.
    • RIXS image reduction - Reduce images to spectra and align them. Also can discriminate single photon events in each image and provides a centroiding algorithm to produce photon counting spectra at higher resolution.
  • New QuickRIXS perspective
    • Allows quick and interactive inspection of RIXS images
    • Can do preliminary processing to plot peak width variations for beamline commissioning
  • General improvements
    • Remove perspectives that have been replaced or are no longer supported (NCD Calibration, etc...)

Bug Fixes -

Many bug fixes went into the release, please refer to JIRA for issues you are particularly interested in.

DAWN 2.7

New features –

  • Improvements to DataVis perspective (http://confluence.diamond.ac.uk/display/DT/Quick+Introduction+to+DataVis)
    • Enhanced line plot stacking/offsetting
    • Ability to colour line plots using image colourmaps
    • Quick image colourmap range and lock setting
    • Quick file open widget now supports ranges e.g. i18-12<35-42>.nxs
    • Better logic for auto selecting data from files, when other files already loaded
    • Dialog to combine multiple line datasets into an image
    • Clean up main toolbar items
  • New plot tools
  • Processing improvements
    • Azimuthal Integration Difference Image – generates image of difference between 2D and remapped azimuthally integrated data for XRD2/SAXS
    • More data plotted in output for visualisation (i.e. a fit in the processing perspective will now plot the residual and fitted function as well as the data)
    • Digital filter to subtract narrow or broad features in line data
    • Multiple Detector Intensity Scaling plugin for beamlines with detectors at varying distances from the sample
    • Added factors to the XPDF processing to adjust the beam polarization and scaling of the Compton scattering.
    • Operation to propagate all axes of dataset
    • Added factors to the XPDF processing to adjust the beam polarization and scaling of the Compton scattering.
    • Deglitch individual and multiple points
  • General improvements
    • Remove some old example data to reduce download size

Bug Fixes -

Many bug fixes went into the release, please refer to JIRA for issues you are particularly interested in.