[NeXus-committee] COMCIF NeXus meeting report and a discussion paper on NeXus interfaces
Mark Koennecke
Mark.Koennecke at psi.ch
Tue Sep 10 15:26:14 BST 2013
Hi,
find attached a draft of:
1) A report on the joint meeting of COMCIFS/NIAC for the nexus mailing list
2) As discussed at the last telco a more fleshed out out suggestion for
NeXus interfaces.
If there are no serious complaints I will post 1) to the nexus mailing
list by the end of the week.
Best Regards,
Mark Koennecke
-------------- next part --------------
NeXus Interfaces
====================
This is an outline how NeXus Interfaces could look like. This is
a basis for discussion. This is not a decided upon NeXus practice.
The Problem
---------------
Quite a number of NeXus base classes have become pretty big. This is due to the
fact that NeXus base classes are dictionaries which hold field names for
all sorts of data items which can be associated with a given NeXus component.
A good example is NXdetector: it holds data items to describe all sorts of detectors,
single, area, arbitrarily shaped, with TOF or without etc. Many users are
confused by NeXus base classes because they mistakingly think that they have to
implement all data items even if they do not make any sense for their application.
A better structure would clearly help here. One way to structure this better would
be inheritance. This would mean introducing more NeXus base classes, for example:
NXsingle_detector, NXarea_detector, NXtof_area_detector etc which form a
hierarchy rooted at NXbeamline_component. The applicable fields for a component
would then be derived by travelling the hierarchy. This comes at the expense
of creating many more backwards incompatible classes. Moreover the inheritance
hierarchy cannot easily be encoded in a HDF5 file. Thus a program looking
for NXdetector would need to know about all its incarnations from an external source.
Which raises issues about maintenability of the external sources.
The second option to be discussed is to use interfaces, like in Java or Go. This
is a separate set of finer grained dictionaries which are used to build up the
NeXus base classes.
NeXus Interface Example for NXdetector
----------------------------------------
How could the interface method look like for NXdetector? To this purpose let us
define a set of interfaces to work this. I use the prefix NXIF here for interfaces
but this is perfectly arbitrary.
----
NXIFbeamline_component
distance
@type=translation
@vector=0,0,1
height
@type=translation
@vector=0,1,0
x_translation
@type=translation
@vector=1,0,0
rotation_angle
@type=rotation
@vector=0,1,0
azimuthal_angle
@type=rotation
@vector=0,0,1
meridional_angle
@type=rotation
@vector=1,0,0
This is an interface meant to be implemented by all beamline components. Its
purpose is to position the component.
----
NXIFmeta
type
description
An interface which contains general meta data about anything
----
NXIFsingle
data
@signal=1
A simple interface for data from a single detector
----
NXIFscanned_single
data[NP]
@signal=1
A single detector scanned. NP is the number of scan points
----
NXIFarea_detector
data[xdim,ydim]
@signal=1
@axes=x_pixel_offset,y_pixel_offset
x_pixel_offset[xdim]
@type=translation
@vector=1,0,0
x_pixel_size[xdim]
y_pixel_offset[ydim]
@type=translation
@vector=0,1,0
y_pixel_size[ydim]
An interface for an area detector. x_pixel_offset and y_pixel_offset describe the grid
of the detector in the detector coordinate system. The origin is the mechanical center
of the area detector. If the pixel sizes cannot be determined from the grid span by
x_pixel_offset and y_pixel_offset, they are given in x,y_pixel_size.
----
NXIFscanned_area_detector
data[NP,xdim,ydim]
@signal=1
@axes=scan_axis,x_pixel_offset,y_pixel_offset
x_pixel_offset[xdim]
@type=translation
@vector=1,0,0
x_pixel_size[xdim]
y_pixel_offset[ydim]
@type=translation
@vector=0,1,0
y_pixel_size[ydim]
An interface for a scanned area detector. The meaning of x,y_pixel_offset etc are
the same as above. One might consider to allow inheritance for Interfaces.
----
NXIFtof_area_detector
data[xdim,ydim,ntbin]
@signal=1
@axes=x_pixel_offset,y_pixel_offset,time_binning
x_pixel_offset[xdim]
@type=translation
@vector=1,0,0
x_pixel_size[xdim]
y_pixel_offset[ydim]
@type=translation
@vector=0,1,0
y_pixel_size[ydim]
time_binning[ntbin]
An interface for an area detector used in time of flight mode.
----
NXIF_arbitrary_detector
data[ndet]
@signal=1
distance[ndet]
@type=translation
@vector=0,0,1
height[ndet]
@type=translation
@vector=0,1,0
x_translation[ndet]
@type=translation
@vector=1,0,0
rotation_angle[ndet]
@type=rotation
@vector=0,1,0
azimuthal_angle[ndet]
@type=rotation
@vector=0,0,1
meridional_angle[ndet]
@type=rotation
@vector=1,0,0
This is an interface to describe a highly irregular detector. A detector which
can only be described by giving full positional and rotational coordinates for
each detector element. ISIS has this kind of detectors.
----
NXIFtof_arbitrary_detector
data[ndet,ntbin]
@signal=1
distance[ndet]
@type=translation
@vector=0,0,1
height[ndet]
@type=translation
@vector=0,1,0
x_translation[ndet]
@type=translation
@vector=1,0,0
rotation_angle[ndet]
@type=rotation
@vector=0,1,0
azimuthal_angle[ndet]
@type=rotation
@vector=0,0,1
meridional_angle[ndet]
@type=rotation
@vector=1,0,0
time_binning[ntbin]
This is an interface to describe a highly irregular detector used in time of
flight mode. The rest is shared with above.
----
To be continued.........................
How to use such Interfaces?
------------------------------
Of course there have to be rules how to use NeXus Interfaces. The rule set is
simple:
1) A group attribute implements is added which is a komma separated list of the
interfaces implemented by the component.
2) The software can then expect the fields defined by the interfaces to appear in
the component class. If this is must or optional is to be discussed.
3) A given NeXus base class can only implement a sensible set of interfaces. It would be
dubious if NXdetector implements NXIFspallation_neutron_source.
An example is in order. Consider:
NXdetector
@implements NXIFbeamline_component,NXIFarea_detector
Then the following fields go into NXdetector:
NXdetector
@implements NXIFbeamline_component,NXIFarea_detector
distance
@type=translation
@vector=0,0,1
height
@type=translation
@vector=0,1,0
x_translation
@type=translation
@vector=1,0,0
rotation_angle
@type=rotation
@vector=0,1,0
azimuthal_angle
@type=rotation
@vector=0,0,1
meridional_angle
@type=rotation
@vector=1,0,0
data[NP,xdim,ydim]
@signal=1
@axes=scan_axis,x_pixel_offset,y_pixel_offset
x_pixel_offset[xdim]
@type=translation
@vector=1,0,0
x_pixel_size[xdim]
y_pixel_offset[ydim]
@type=translation
@vector=0,1,0
y_pixel_size[ydim]
Another example: a scanned single detector
NXdetector
@implements NXIFbeamline_component, NXIFscanned_single_detector
distance
@type=translation
@vector=0,0,1
height
@type=translation
@vector=0,1,0
x_translation
@type=translation
@vector=1,0,0
rotation_angle
@type=rotation
@vector=0,1,0
azimuthal_angle
@type=rotation
@vector=0,0,1
meridional_angle
@type=rotation
@vector=1,0,0
data[NP]
@signal=1
Advantages of NeXus Interfaces
--------------------------------
What would be the advantages of the NeXus interface approach:
* We can be far more specific what goes into a base class for a
use case then with the current base class description.
* We can do so without cluttering the namespace with even more base classes.
* User confusion is reduced.
* The approach is easily extended to new use cases by defining a new interface
for the new use case.
* The interface approach is backwards compatible. We add to NeXus rather then defining
something entirely new. Old files can be updated to the way of the interface by
adding required fields and the interface group attribute.
Disadvantages of NeXus Interfaces
------------------------------------
* It is yet another concept and set of rules to teach and learn.
NeXus Interfaces and Mapping to CIF
---------------------------------------
When mapping between NeXus and CIF there is a major difficulty: the way
how multiples are handled in CIF and NeXus. Consider an instrument with
two detectors. In NeXus this would map to two NXdetector classes and two
NXdata classes with different names within the hierarchy.
In CIF one would loop over detector name and all the fields of the base
class. A loop is basically a table. The above example would thus map to a
table with detector_name and all the fields of the base class as columns
and two rows indexed by detctor name.
With the current size of the NeXus base classes, this would make for
unwieldly and sparsely populated tables. With NeXus Interfaces this becomes
much more workable. Each NeXus interface would map to a CIF category(table) and
there would be other categories (tables) which detail the list of components
of the instrument and which interfaces are implemented by each component.
Herbert solved the problem in his concordance document by appending the NeXus
base class name and the component name together. But this makes for long and
unpredictable CIF category (table) names.
Where To Go From Here?
-------------------------
1) All the confusion generated by this docuemtn must be resolved.
2) The big thing is: do we want NeXus Interfaces? This requires a vote.
3) Which rules do we use to write NeXus Interfaces. I made something up for the
this example. But, of course, this could be different.
4) How do we document NeXus interfaces? NXDL would be the first call, but.....
5) Polishing up NeXus interfaces most likely requires a code camp.
-------------- next part --------------
Report on the Joint NeXus COMCIFS Meeting
============================================
Introduction
--------------
Before the ECM meeting 2013 at Warwick University there was
a meeting between representatives from COMCIFS, the working group
which oversees the CIF data format, and representatives of the
NIAC. The purpose of this meeting was to find out how NeXus and CIF
can learn from each other and how they can interoperate better.
This meeting took place at Warwick university on Thursday,
August 22, 2013. NeXus was represented by Tobias Richter, Ben Watts,
Freddie Akeroyd, Eugen Wintersberger, Herbert Bernstein and
Mark Koennecke. COMCIFS was represented by James Hesters, Brian McMahon,
John Westbrook, Brian Toby, Jonathan Sloan and Herbert Bernstein.
Meeting Agenda
-----------------
The meeting agenda was pretty informal:
# First COMCIFS representatives gave an overview of CIF
# Mark Koennecke gave an introduction to NeXus/HDF5
# A discussion about how NeXus and CIF can collaborate
Meeting Results
------------------
The atmosphere at the meeting was quite open and friendly. There was an understanding
that the new generation of X-ray detectors generate big data. And that big data
requires HDF5. COMCIFS appeared to be happy to let facilities store raw data
in NeXus/HDF5. Brian McMahon pointed out that the acronym CIF really stands for
Crystallographic Information Framework which leaves room to have different data
formats for different purposes.
In terms of a collaboration between COMCIFS and NeXus it was agreed to leave
CIF be CIF and NeXus be NeXus for the time being. The collaboration between COMCIF
and NeXus will take two main directions: we will continue the work started by
Herbert Bernstein et al to create a mapping between CIF/CBF and NeXus. The
other direction is to develop crystallography related NeXus application definitions
together with COMCIF. The start will be an application definition for protein
crystallography. However there is potential to codevelop application definitions
for a whole slew of techniques. It was agreed to have a joint COMCIFS/NIAC
teleconference in three months time in order to review progress and to coordinate.
Many Thanks to Brain McMahon for organising this meeting
More information about the NeXus-committee
mailing list