[Nexus] Desire for a more rigorous NeXus standard

[Tom Kozlowski]

The 8 LANSCE instruments running with the our data acquisition system
archive data using the NeXus standard.  Some of our instrument users wish to
use the Argonne ISAW tools (based on the NeXus standard). In trying to
process our NeXus files with ISAW, we encountered problems due to differing
interpretations of the NeXus standard. I doubt this is an infrequent
occurrence, now that several institutions are involved in writing and using
NeXus files. Varied interpretations of the standard as it currently exists
will inevitably result in incompatibilities
between NeXus based tools and NeXus files produced at different facilities.

Now that the NeXus community has reached this point of maturity, it is a
good time to refine and more formally document the NeXus standard,
minimizing ambiguities and making sure it meets the needs of implementers
and users.  The meta-DTD format currently on the NeXus website is a good
step in this direction, but more is needed.  In the near future, we would
like to see a formal document produced that unambiguously defines the NeXus
standard. From my experiences in this sort of activity, one way to move
quickly to achieve this is a process like the following:

  * Organize and convene a working group (volunteers from major
    NeXus institutions). The working group needs some basic
    organization (defined chairperson, editor, and maybe a webmaster).
    (I don't think slow evolution of the current NeXus website in
    conjunction with random message exchange will result in a useful
    formal NeXus definition within a sufficiently short time.
    A working group that can carry out more intense discussions and
    work is really necessary.)

  * The working group creates a draft document of a formal definition
    of the NeXus standard, as currently understood. This process itself
    will inevitably result in unresolved issues (ambiguities, questions
    and suggestions from various quarters). The draft should explicitly
    list these unresolved issues. It may also may be appropriate to
    include "standard" instrument definitions in this draft.

  * Send the draft document to the NeXus mailing list for comments,
    especially in regard to the list of issues.

  * Incorporate the comments into the draft document.

  * The working group examines the draft document, discussing in detail
    the standard and all issues, attempting to resolve issues. Difficult
    issues may require working group votes. In my experience,
    a consensus usually is acheived on most issues.

  * The working group creates an official NeXus Standard (Version 1.0)
    document, reflecting the resolved issues.

  * Maintain the standard.  Issues will naturally arise over time
    as the standard is used in different environments.  These need
    to be documented for later action.  Periodically this whole
    process is repeated and results in a new version of the standard.
    The maintenance can be done by a formal board with indentified
    representatives from major institutions, or perhaps by the less
    formal organization of a working group (those who have
    volunteered to do the work).  I think in small communities like
    ours, the latter works better, but it still needs some basic
    structure.

To help us resolve differences between LANSCE and ISAW NeXus interpretation,
several of us at LANSCE (Gary Cooper, Tom Kozlowski,
Thomas Proffen) went over the meta-DTD version of the NeXus standard. We
assembled our questions, comments and suggestions in a document.   It is
organized in terms of NeXus classes, and includes a list of "issues" for
each
class and also global issues. This document is an example of -- and with
some work could become the basis for -- a draft of a more formal NeXus
definition.  The document can be accessed at the following URLs:
 http://strider.lansce.lanl.gov/canps/nexus/nexus_spec_xml_1.doc (Word)
 http://strider.lansce.lanl.gov/canps/nexus/nexus_spec_xml_1.pdf (PDF)

It would be useful to discuss these issues and this proposal with other
interested parties at NOBUGS 2002. If there is some kind of
consensus, we can attempt to begin the organization of a working group.
What do you think?

Regards, Tom

Thomas Kozlowski
LANSCE-12 Los Alamos Neutron Science Center
Email: kozlowski_thomas at lanl.gov
Voice: 505-667-7747  Fax: 505-665-2676