[NeXus-tech] RFC Summary

[Mark Koennecke]

=============================================
Summary on Comments for variants and errors
=============================================

Some weeks ago we asked for comments on two controversial topics,
variants and errors, from the wider NIAC. This now  is a summary off
the additional options which came in by email, together with the original
text. This summary serves as material for the next Telco and to reboot
the discussion.

Variants
==========

The problem: there are parameters in a NeXus file for which better
values may become available later on through a refinement, calibration
or whatever. The requirement is to store both the old and the new value
together with a time stamp. Variants are required for a full mapping
of CIF concepts to NeXus. The experience of the CIF people is that
variants will be used only occasionally and only for a few fields.
Thus a heavy weight scheme for handling this situation is not necessary.

The suggestion: Apply the following process when a new value is available
for an existing field in an NeXus file:
1) memorize the current value
2) Update the field and assign a time_stamp attribute to it.
3) Store the old value in a new field with a name derived by appending
    _variant to the original field name. Assign a time_stamp attribute 
to this
   new field too.

When more then one variant is required, build names by appending 
_variant_somename
, _variant_someothername to the original field name.

An example:

wavelength
   @time_stamp=sometime
wavelength_variant
   @time_stamp=someothertime

A more elaborate example:

wavelength
   @time_stamp=sometime
wavelength_variant_asdialled
   @time_stamp=someothertime
wavelength_variant_fromSI
   @time_stamp=someothertime



Discussion
------------

We considered using a variant attribute to the field. But this causes 
problems
when a larger field, like for example an area detector efficiency 
correction,
needs a variant.  It is also non obvious how to store the time stamp then.

We are aware of the fact that the suggested scheme can lead to long names.
We rely on the fact that the CIF experience is that variants are rarely 
used.
And CIF has been around for a while. Reading software only needs to concern
itself with the real field name which is supposed to hold the best 
available
value.

Comments
------------------

Ben did not like this and argues for putting those variants a level deeper
in the hierarchy like:

wavelength
   @time_stamp=sometime
wavelength_variant/
   wavelength
   @time_stamp=someothertime

A more elaborate example:

wavelength
   @time_stamp=sometime
wavelength_variant/
   wavelength
   NXnote = asdialled
   @time_stamp=someothertime
   wavelength_variant/
     wavelength
     NXnote = fromSI
     @time_stamp=someothertime

I assmume that the groups are of class NXvariant. There is a problem in
the second example as NeXus (and HDF5?) do not allow two groups of the
same name.

 From the CIF community, represented by Graeme Winter and Aaron Schwartz 
there
was aconcern to keep the CIF details field. Which conveys more 
information on
the variant. They seemed happy when another field is added when necessary.

I am not so happy to have an NXvariant group. NXvariant will be one of 
those
groups which is ill defined as it can hold almost everything.


Standard Deviation
=====================


The problem: Real physical values have errors. NeXus accounted for this by
having fields with _error appended for a few fields. The reqirement now is
to have a standard scheme for marking up experimental errors for any field.

The suggestion: store the errors of a field value in a new field. The name
of this field is derived from the original field name by appending _esd to
it. Example:

wavelength
wavelength_esd

Comments
---------------

Peter Peterson commented that we more or less aggreed to use the appendix
_error for errors. Like in:

wavelength_error

Pete Jemian, on behalf on canSAS, added that there may be more complicated
cases of errors which may require multiple different error fields. His 
suggested
solution is to have an uncertainties attribute which names the fields in 
the same
group which actually hold the errors. For example:

wavelength
   @uncertainties=wavelength_errors, wavelength_stddev
wavelength_errors
wavelength_stddev

The case made for this is to describe uncertainities resulting from 
different sources
or determined in different ways. Which basically is a variant on the 
error......



Best Regards,

          Mark Koennecke