[NeXus-tech] SAS2012 canSAS data format presentation

[Pete Jemian]

On 11/8/2012 4:55 AM, Tobias.Richter at diamond.ac.uk wrote:
> When we were discussing this with the NeXus people for the more general case, they preferred @axes over @I_axes
> Is that something we want to address and change? On the technical side it makes no difference for canSAS, I believe but would offer a wider compatibility (potentially).

Sorry, long answer coming...
This change involves how multi-dimensional data is described.

The answer from the NeXus tech committee on this is not satisfying to me 
but they may not have been able to see our use so clearly.  I believe 
our plan is superior and should not be changed so quickly as the real 
benefit would be to NeXus, rather than canSAS.  We should press them to 
consider this better, by making a formal proposal to them and making 
that proposal very clear.

Our required dataset in SASdata is "I" and we created an @I_axes 
attribute to describe the components of its various axes.  One could use 
this same construct to describe the axes of any dataset (such as 
"background" would have an associated @background_axes attribute on the 
SASdata group).  The NeXus Tech Committee suggested that we change the 
name of this attribute to "axes" to be more consistent with current 
NeXus usage.

The NeXus tech committee is interested in making changes that will not 
break backward compatibility and preserve existing structures.  One of 
them commented "Currently we have two ways and this would be a third." 
Understood.  See this link for the NeXus algorithm (incorporates both 
current methods) to find the plottable data:
    http://download.nexusformat.org/doc/html/datarules.html#index-27

Another relevant link in the NeXus manual:
 
http://download.nexusformat.org/doc/html/datarules.html#linking-multi-dimensional-data-with-axis-data

I believe ours is a change that NeXus should give more consideration, 
especially since it offers a solution to a problem they have.

In NeXus, there is an @axes attribute possible for any dataset.  In our 
plans, we move this attribute to the parent group of the dataset and we 
have sound reasons for this change, involving linked datasets.
Also involved in this change is how NeXus uses the @signal attribute on 
datasets.  What will be hard for NeXus to adopt is the notion that they 
should modify the technique they currently use to identify the principal 
data to be plotted, since that will certainly mean they have three ways 
to do describe the axes of multi-dimensional data.  The proposed plan 
for them addresses directly a known limitation in a positive and 
flexible way.

Consider this example from the NeXus documentation:

   data:NXdata
     time_of_flight:float[250] = 1500.0 1502.0 1504.0 ...
     polar_angle:float[100] = 15.0 15.6 16.2 ...
     some_other_angle:float[100] = 0.0 0.0 2.0 ...
     data:float[100,250] = 5 7 14 ...
       @axes = polar_angle:time_of_flight
       @signal = 1

Using the canSAS ideas, we would re-write this:

   data:NXdata
     @signal = data
     @data_axes = polar_angle, time_of_flight
     time_of_flight:float[250] = 1500.0 1502.0 1504.0 ...
     polar_angle:float[100] = 15.0 15.6 16.2 ...
     some_other_angle:float[100] = 0.0 0.0 2.0 ...
     data:float[100,250] = 5 7 14 ...

Note that the name of the @axes attribute changed to @data_axes as it 
moved up to the NXdata group.  (One could argue changing the name of 
@signal to @plot but let's leave that discussion to NeXus).

Q: Doesn't look so different so why should NeXus change?
A: When using hard links of data sets into other groups.

    In HDF5, a hard link of a dataset will represent the
    dataset name, its data, and its attributes in more
    than one HDF5 group.

    Consider a NeXus NXdata group, one of the datasets
    must have the attribute @signal=1.  That's a NeXus rule.
    If a second dataset were to be linked into this
    dataset (conceivably from another NXdata group), it
    might also have the @signal=1 attribute set.  This
    would violate the NeXus rule since two datasets in
    the same NXdata group would have the @signal=1 attribute.

    We could /easily/ run into this by linking into the
    same SASdata group some additional (non-required)
    experimental data such as dark_frame, white_frame,
    or background.

Our plan completely avoids this problem and fixes it for NeXus.
(Also, our plan assumes the SASdata group attribute @signal = I)

Thanks for hearing me out,
    Pete


-- 
----------------------------------------------------------
Pete R. Jemian, Ph.D.                <jemian at anl.gov>
Beam line Controls and Data Acquisition, Group Leader
Advanced Photon Source,   Argonne National Laboratory
Argonne, IL  60439                   630 - 252 - 3189
-----------------------------------------------------------
    Education is the one thing for which people
       are willing to pay yet not receive.
-----------------------------------------------------------