[NeXus-committee] obsolescence and evolution: NeXus needs a historic dimension

[yayahjb]

Dear Colleagues,

   Looking back at the past half century, one thing I am certain of is 
that there will continue
to be new, exciting and wildly different way of collecting scientific 
data that will be developed.
What we need it a framework that can continue to adapt and evolve, where 
an scientist can
use what he/she finds useful from what has already been done, but feels 
free to add
new groups/tags/fields, etc. that allow him/her to accurately describe 
that way in which his/her
new addition science is being done.

   We need to be descriptive, rather than prescriptive.  We need to 
accurate describe what
people are doing now and in the future we will need ways to allow our 
future selves to
accurately describe what we are doing then.  No set of 
groups/tags/fields we come up with
now will be complete and final.  It will just be the best we can do now 
on the basis of the
information we have now.

   For anything, and object, we work with, there will be some general 
descriptive name that
give the "type" or "class" of that object, such as NXdetector.  That 
name does not, necessarily
tell us which particular object we are dealing with.  That needs another 
name for the particular
instance of the class, e.g. "detector" or "Pilatus 6M SN 1342" or 
"mugwump".  With experiments
gathering increasing numbers of interesting and useful objects used to 
do a particular experiment,
we need to allow for an arbitrary number of instances of classes to be 
gathered together in
on NeXus entry -- and that mean allowing for multiple detectors, 
multiple samples, multiple
beams, all of which need both class names and need object instance 
names.  I urge that we
move way from all the fixed uses of object instance names, such as 
"data", "detector", "instrument"
and "sample" and allow the use of arbitrary names with meaning to the 
experimenter.

   This can create a problem with fields, which, in NeXus, have 
classically been identified by their
name, rather than by their type.  But if you look at the HDF5 
implementation of NeXus classes,
they are just named groups with an NX_class attribute.  We can easily 
remove the restrictions
on the naming of fields in a very general way by adding a new attribute 
NX_field_class with
a value equal to the current pre-defined field name.  Then, if an 
NX_field_class attribute is specified
for a field, that attribute value would be used for validation instead 
of the name.  If no NX_field_class
attribute is specified the name would be used for validation.

   This would, for example, immediately resolve the problem of dealing 
with the multiple data fields
we are increasingly needing to deal with in an NXdata class instance, 
replacing the limited

     data:NXdata
       data_00001=[.....]
       data_00002=[.....]
   ...

with the more general

    mywonderfuldata:NXdata
       first_interesting_output=[...]
          @NX_field_class="data"
       second_interesting_output=[...]
          @NX_field_class="data"

Regards,
     Herbert




On 7/17/14 11:24 AM, Osborn, Raymond wrote:
>> Those tags should as much as possible be compatible with each other. Obviously "event mode" and "monochromatic beam" are unlikely to occur together. But rather than prescribing a name for groups ("detector") we'd have tags for saying "I'm the small angle detector" and "I recorded the fluorescence".
>>      
>
> Actually, event mode using monochromatic beams is the basis for one of the most exciting recent developments in inelastic neutron scattering, i.e., measuring 4-dimensional S(Q,w) while continuously rotating the sample in a monochromatic beam. This perhaps highlights the challenge of keeping the NeXus base classes current - we really need constant input from the whole community because new experimental configurations are continually being developed. There is a feedback mechanism in facilities that have centralized software development. The Mantid developers are aware of the 4D-S(Q,w) technique because they are developing data reduction for it. It may be more difficult for synchrotron beamlines that are responsible for their own software.
>
> Ray
> P.S. This is another reason for allowing ad hoc use of NeXus while people are getting experience with the best way of handling data collected by a new technique. We shouldn’t fix the standard before data analysis procedures are mature.
>