[Nexus-developers] NeXus Quo Vadis

Mon Feb 20 16:51:18 GMT 2012

Hi there,

Just to clarify my position and that of the ESRF concerning what Mark 
has written:

"""
NeXus-Armando-Light
A better name is required here. The Armando is for Armando Sole who more 
or less suggested this
aproach if I read him correctly. This is another version of a lighter 
NeXus. I believe the approach
suggested here would formalize a practice of using NeXus which is common 
at synchrotron sources. This
version looks like this:
1. Use HDF-5 as a file format
"""

100 % correct.

"""
2. Store your data in a simplified hierarchy using NXentry and 
NXcollection groups. Apply a lot of
freedom how you do this.
"""

NXentry mandatory.
NXcollection for anything not dealt with by other NXclass.

If you want to describe your beamline, then there is a group 
(NXinstrument) for that. I would not write it into an NXcollection.
Personally I like how ALBA is generating Nexus files.

"""
3. Have NXmethod groups at NXentry level which contain links to those 
data items required for a
specific use of the file. For example a group NXsas would contain links 
to all data items required for
small angle scattering analysis. The content of the NXmethod groups can 
be derived from the
existing application definitions. Just flatten the application 
definition hierarchy and resolve name
conflicts by prefixing.
"""

That is certainly not our point of view. At last NIAC was decided to 
have NXsubentry performing that role. I would have preferred to have 
NXdefinition as NX_CLASS attribute, but at the end is just a personal taste.

What we advocate is, for instance:

/(root)
/whatever_entry_name:
      @NX_CLASS:NXentry
      start_time
      end_time
      ....
/whatever_entry_name/whatever_group_name:
     @NX_CLASS:NXsubentry
     definition  (string set to the particular technique implemented 
helping to understand the rest of the fields in the subentry)
     .....

That approach solves the issue of several techniques under the same 
NXentry because you can have several NXsubentry groups.

Our most common case is simultaneous fluorescence-diffraction. That 
approach is compliant with NeXus and it is not against any NeXus philosophy.
What we do not accept, is to force the different items to be found 
inside the NXsubentry group to belong a particular place in the NeXus 
tree. They just need to be there, either as datasets, as groups or as 
links to datasets or groups. Please take a loot at the attached talk I 
gave at the Q2XAFS 2011 workshop.

There are currently initiatives at APS and SLS among others for 
standardizing tomography data. They define a exchange group with the 
information needed to analyze the data, with other optional groups to 
describe the instrument, and XML representation for validation purposes 
and so on. When I take a look at their work, they are doing nothing 
different to what can be done with NeXus adding an attribute NXsubentry 
to their exchange group. Should one congratulate the NIAC for 
encouraging such practices with the NIAC intransigence?

There are several consequences of the "Armando's approach":

1 - If HDF5 is the format, the NeXus API is not necessary.
2 - If the analysis information is in the definition, SOLEIL's common 
data model is already contained in it and becomes redundant. Its only 
motivation would be to deal with non-HDF5 legacy formats.
3 - The NeXus application definition validation tool is a perfectly 
valid tool.

Other big mistakes are that the NIAC has tried to answer:

a) how one had to store the data and
b) how one had to analyze them

It is our opinion that:

a) How to store the data is a question that belongs to the data 
acquisition people of the facility and the constraints of the experiment 
(speed)

b) How to analyze the data belongs to the community employing a 
technique and/or the analysis application developers to decide what is 
needed. The NIAC can define a set of required data to perform an 
analysis, but if the leading applications decide to use other parameters 
the NIAC has nothing to do. As a matter of fact there are community 
initiatives going on for XAS and for tomography. If they arrive to a 
consensus, the NIAC will have *no other choice* than to accept them as 
they come.

I think NeXus can still play a role:

1 - Acting as custodian of the community approved application 
definitions and providing the corresponding validation tool(s).
2 - Presenting anything else as a set of recommendations.
3 - Redirecting NAPI maintenance/development efforts to work with the 
HDF group to  implement much needed features as the single writer 
multiple reader and to push for interesting developments that currently 
are just proposals. As a matter of fact, there is a proposal for an 
abstract object layer that would allow to use HDF5 on non HDF5 files 
therefore getting all SOLEIL's common data model objectives directly 
supplied by the HDF5 library.

Best regards,

Armando

On 20/02/2012 16:32, Mark Koennecke wrote:
> Hi,
>
> as promised at the last tele conference I send a short document about 
> some views I
> have about NeXus and the future of NeXus. It questions many of the 
> things we have done
> so far. So this may be a bit disruptive. My general ideas was to 
> discuss these topics at
> the next NIAC. But I first want your feedback. Depending on this:
> - we can decide to bury the document
> - we can improve it to become a framework for discussion at NIAC
>
> I feel a little uneasy about writing such a document. On the other 
> hand, if times change
> NeXus may need to change too.
>
> This paper is for NIAC members only.
>
> Best Regards,
>
>               Mark Koennecke
>
>
>
> _______________________________________________
> NeXus-developers mailing list
> NeXus-developers at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus-developers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nexusformat.org/pipermail/nexus-developers/attachments/20120220/737b833c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SOLE-HDF5NeXusAndBeyond.pdf
Type: application/pdf
Size: 1474632 bytes
Desc: not available
URL: <http://lists.nexusformat.org/pipermail/nexus-developers/attachments/20120220/737b833c/attachment-0001.pdf>