[NeXus-committee] obsolescence and evolution: NeXus needs a historic dimension

[Pete Jemian]

On 7/16/2014 9:48 AM, Joachim Wuttke wrote:
> Are raw NeXus visualization tools actually used by instrument
> scientists?  Where?

* PyMCA is often my first suggestion for raw GUI visualization.
* HDFview is the most raw GUI tool, good for cursory inspection.
* h5dump is good for inspection of content as text.
* h5toText is good for inspection of structure as text.
* NeXpy is a GUI tool that is not only aware of NeXus
   structure, but relies on it.  You can double-click the
   NXentry group and a default visualization will be
   generated.  That is enabled by an implementation of the
   strategy to find the plottable data given in the manual [1]:
 
http://download.nexusformat.org/doc/html/datarules.html#find-plottable-data

All of these are general purpose tools used at APS, in addition to other 
custom software.

1. [note for documentation editor:
strategy to find the plottable data is also given here:
 
http://download.nexusformat.org/doc/html/design.html#nxdata-facilitates-automatic-plotting 

resolve these two and present it only once in the manual, reference from 
the other place]

 > Why aren't they running their raw-data through
> some method-specific reduction?

Now, you are talking about data files with more than just the required 
content.  And the goal has changed from visualization to method-specific 
reduction.

Consider this hypothetical.  Reduction might need the monochromatic 
wavelength (not a TOF instrument).  The contract from NeXus is that the 
wavelength is provided on this general path:

/: NeXus HDF5 data file
   <name>:NXentry
     <name>:NXinstrument
       <name>:NXmonochromator
         wavelength:NX_FLOAT
           @units="<NX_WAVELENGTH>"

Here, <name> is not specified by NeXus (the community has told us they 
want this flexibility to choose the HDF5 names of NeXus groups). 
However, the NeXus group can be determined by looking at the NX_class 
attribute of the HDF5 group.  Thus, the contract is that the instrument 
wavelength is spelled "wavelength" (and not Wavelength or lambda or 
...), the value is a floating point number, and the supplied units are 
"any valid wavelength units" with the suggestion from the documentation 
of "example: Angstrom".  Further to the contract, the wavelength (if 
present) will be presented on the class path shown above.  The standard 
for base classes does not say the wavelength must be present, it only 
defines how it will be spelled, where it will be stored, and what is the 
data type and the type of units.

Using more fields and base classes, the description of an instrument can 
become more rich.  And more specific to that particular implementation 
of that instrument.

For example, a hypothetical reduction method may compute Q from the raw 
data.  With monochromatic wavelength, one data reduction method would 
need 2theta.  Here's an example of where to expect 2theta: 
http://download.nexusformat.org/doc/html/design.html#links

However, such hypothetical method-specific data reduction must agree on 
the name of the 2theta data, and its location.  This is done by 
specification in an application definition.

One must presume instrument scientists and/or their users _always_ run 
their raw data through method-specific (and current instrument 
implementation specific) reduction methods.  The might or might not use 
generic visualization tools.  Those instrument-specific methods may 
provide visualization.  As you have identified, there are always 
differences between two implementations of the same instrument.  Even 
the same instrument can be changed from user to user.
Then, after reduction, scientists apply methods appropriate to the 
scientific analysis of interest.

Pete