[NeXus-committee] RFC variants and esd

[Pete Jemian]

A discussion of representing uncertainties came up at the last canSAS 
workshop.  Here is documentation from that discussion describing 
representation of various experimental uncertainties:

http://www.cansas.org/formats/canSAS2012/1.0/doc/examples.html#complicated-uncertainties

Fundamentally, a field would have an attribute that names a child group 
in which the uncertainty (single or plural) is described with fields. 
Thus, the question of how to name is passed to the one who writes the 
HDF5 file.  The question comes up when multiple uncertainties must be 
documented and which is the one to use.  The canSAS document describes 
one possibility.

notes from the group discussion are here (search for text "uncertain"):

http://www.cansas.org/wgwiki/index.php/2012_Data_Discussion

Pete

On 10/24/2013 8:34 AM, Benjamin Watts wrote:
> Hi Everyone,
>     I'm fine with the "standard deviation" proposal. I don't much like
> the the "variants" proposal though because I would rather add another
> level to the hierarchy than extend the name. That way, if further
> variants get added, the previous values can be seen by following the
> string of 'fieldname_variant' groups, with the original recorded value
> found with the longest string of 'fieldname_variant' groups. Rewriting
> Marks examples:
>
> wavelength
>    @time_stamp=sometime
> wavelength_variant/
>    wavelength
>    @time_stamp=someothertime
>
> A more elaborate example:
>
> wavelength
>    @time_stamp=sometime
> wavelength_variant/
>    wavelength
>    NXnote = asdialled
>    @time_stamp=someothertime
>    wavelength_variant/
>      wavelength
>      NXnote = fromSI
>      @time_stamp=someothertime
>
>
> Cheers,
> Ben
>
>
> On 24/10/13 14:56, Mark Koennecke wrote:
>>
>> Request for Comments
>> =======================
>>
>> There are two topics on which we seek comments from the wider
>> NIAC. Both issues arose from the discussion about the CIF-NeXus
>> mapping and a MX application definition in collaboration with the
>> CIF group. The issues were discussed at length at various NeXus
>> teleconferences. Now we seek comments from the wider NIAC in
>> preparation for a proposal to vote on.
>>
>>
>> Variants
>> -------------
>>
>> The problem: there are parameters in a NeXus file for which better
>> values may become available later on through a refinement, calibration
>> or whatever. The requirement is to store both the old and the new value
>> together with a time stamp. Variants are required for a full mapping
>> of CIF concepts to NeXus. The experience of the CIF people is that
>> variants will be used only occasionally and only for a few fields.
>> Thus a heavy weight scheme for handling this situation is not necessary.
>>
>> The suggestion: Apply the following process when a new value is available
>> for an existing field in an NeXus file:
>> 1) memorize the current value
>> 2) Update the field and assign a time_stamp attribute to it.
>> 3) Store the old value in a new field with a name derived by appending
>>    _variant to the original field name. Assign a time_stamp attribute
>> to this
>>   new field too.
>>
>> When more then one variant is required, build names by appending
>> _variant_somename
>> , _variant_someothername to the original field name.
>>
>> An example:
>>
>> wavelength
>>   @time_stamp=sometime
>> wavelength_variant
>>   @time_stamp=someothertime
>>
>> A more elaborate example:
>>
>> wavelength
>>   @time_stamp=sometime
>> wavelength_variant_asdialled
>>   @time_stamp=someothertime
>> wavelength_variant_fromSI
>>   @time_stamp=someothertime
>>
>>
>>
>> Discussion
>> ------------
>>
>> We considered using a variant attribute to the field. But this causes
>> problems
>> when a larger field, like for example an area detector efficiency
>> correction,
>> needs a variant.  It is also non obvious how to store the time stamp
>> then.
>>
>> We are aware of the fact that the suggested scheme can lead to long
>> names.
>> We rely on the fact that the CIF experience is that variants are
>> rarely used.
>> And CIF has been around for a while. Reading software only needs to
>> concern
>> itself with the real field name which is supposed to hold the best
>> available
>> value.
>>
>>
>>
>> Standard Deviation
>> ----------------------
>>
>>
>> The problem: Real physical values have errors. NeXus accounted for
>> this by
>> having fields with _error appended for a few fields. The reqirement
>> now is
>> to have a standard scheme for marking up experimental errors for any
>> field.
>>
>> The suggestion: store the errors of a field value in a new field. The
>> name
>> of this field is derived from the original field name by appending
>> _esd to
>> it. Example:
>>
>> wavelength
>> wavelength_esd
>>
>> Please let us know what you think. Other suggestions, even completely
>> different
>> ones, are very welcome. The only requirement is that the suggestion
>> offers a
>> solution to the original problem.
>>
>> Best Regards,
>>
>>          Mark Koennecke
>>
>> _______________________________________________
>> NeXus-committee mailing list
>> NeXus-committee at nexusformat.org
>> http://lists.nexusformat.org/mailman/listinfo/nexus-committee
>
> _______________________________________________
> NeXus-committee mailing list
> NeXus-committee at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus-committee