[NeXus-committee] Concordance paper, ECM

Mon Apr 22 11:43:59 BST 2013

Hi all,

Just to reiterate what Herbert has written in the document: This draft is a work in progress. In particular this version is one that I didn't have time to comment on. 

The idea behind this whole operation is to make a minimum of changes in either format. If we need to preserve information that currently cannot be expressed we add a place for it. Everything else should go into existing slots.
In rare cases were the existing slots are not suitable, we may need to make amends. But I would say we clearly haven exhausted our options yet. That's why we ask for comments.

In line with the NeXus philosophy there a number (potentially optional) fields and attributes may be added to aid a faithful roundtrip conversion between NeXus and CBF.  There is  a fair number of fields from that category and where we aren't mapping correctly yet, it would probably we helpful to identify which is which.

Just one comment on a few points raised by Mark (leaving the major share for Herbert):

Section 3.2, P 10, Admin Feature Request
---------------------------------------------------------
We already have experiment_identifier and entry_identifier defined. With entry = scan (more or less) we would only need a diffraction_identifier (to keep the same naming convention), which in my understanding would be a per-frame array. Right?

Section 3.16, P36, Variant feature
----------------------------------------------
NXsubentry addresses a different problem. You might also want to record a variant of something that is in a subentry. 
My current idea is that of a NXvariant group that inherits from NXentry and also live at the root level. It would reference the NXentry or other NXvariant it operates on (in e.g. an attribute) and would only carry the item at are changed and the required structure to hold them. 
An alternative would be to link the unchanged entries. It keeps reading the variant simple in software but that hides the purpose of that feature and makes the changes less accessible to humans.

I am not sure I see much scope for discussing this with the CIF committee. I am not even sure they have seen this document yet. 

Interest for doing anything in August was low in the Doodle poll. We have some discussions around the ECM meeting if people change their mind. For the moment I do have Jonathan devoted to the project. We have to keep making progress. 

Regards,

Tobias

________________________________________
From: nexus-committee-bounces at nexusformat.org [nexus-committee-bounces at nexusformat.org] on behalf of Mark Koennecke [Mark.Koennecke at psi.ch]
Sent: 22 April 2013 10:14
To: Discussions of the NeXus Advisory Committee
Subject: [NeXus-committee] Concordance paper, ECM

Hi everyone,

There are two things which we need to be decide upon rather quickly
in order to allow planning to get done.

The first is that it occurred to me that merging NeXus and CIF might
mean that we need to understand their dictionary technology. So may be
a few of us should ask to join the CIF workshop, August 23-24. I myself
would be interested. Anyone else?

The second is that Herberts concordance paper raised a number of issues.
Thus I wonder if we should not have a 1 day code camp/meeting before
meeting with the CIF people to sort this out. Anyone opinions on this?

I also took a little time to look through the concordance paper. Find my
comments attached below.

Best Regards,

                  Mark Koennecke

  Notes on Herberts Concordance Summary: 4/2013
================================================

Herberts paper raises a few issues and asks for some new features in NeXus.
This is my humble attempt to summarize and comment the issues.

Section 2, Page 2
--------------------

I was a little confused: As I see it this comes down to:
CBF uses tables, NeXus hierarchical groups. The basic mapping is
a CBF table to a NeXus group. CBF table columns map to NeXus fields.
In many cases a CBF table has only one row. If there are multiple rows
in CBF for example, NR,then in NeXus the fields will be
arrays of length NR.

Page 3: During the NeXus Telco this was resolved: different scans map to
different NXentries/ files.

Section 3.2, P 10, Admin Feature Request
------------------------------------------

This is a feature request: especially in PX there is a deeper level
hierarchy of experiments: there is the study, there are multiple crystals
(of the same protein) and there are multiple scans on those crystals. In order
not to loose this hierarchy it is needed to include this administrative
information in each file. Herbert suggest 3 additional fields or attributes:
scan_id, diffrn_id, entry_id at NXentry level. I personally prefer fields.

Section 3.3, P10, Multiple detectors
-----------------------------------------

NeXus deals with multiple detectors or ROI's by storing data in
separate NXdetector groups. Which makes a lot of sense. Q: Why does this
fall over?

Section 3.6, P12-13, NXdetector fields
-------------------------------------

This asks for some new fields in NXdetector.

Section 3.7.1, P14, Array_Structure_List
------------------------------------------

I compared this with the CBF description. This is about array order and dimensions etc.
IMHO, NeXus covers this by the HDF-5 dimensions and the offset and stride attributes.

Section 3.7.2, P15,  array_structure_list_section, ROI feature missing
-------------------------------------------------------------------

Again looking at the CBF documentation this seems to declare a region of interest into
a larger array. NeXus does not have this per se. Currently I fail to understand how this
is used: will there be another data array? Or does this mean: I have this big fat array,
but just use a smaller part of it?

Section 3.7.3, P 15-16, array_structure_list_axis, CBF-NeXus difference
-------------------------------------------------------------------------

This is about associating scan axis with the data array. CBF does this by stating
a lot of parameters: starts, increments etc. for each axis. Or appears to be doing
this. I might have misread the CBF docs. In NeXus this is solved by storing actual
positions along the scan in arrays and the association is done by the scan rules,
most notably the population of NXdata in the scan case. This actually may require some
discussion: IMHO, storing arrays is better as it allows for the possibility of
mispositionings.

Section 3.8, Axis, Coordinate System Features
--------------------------------------------------

Here is a  major difference: CBF supports multiple coordinate systems.
Correspondingly CBF has a way to document the coordinate system used by defining the
new coordinate system in terms of the standard laboratory system and by defining the
directions of gravity and the beam.

NeXus does not do this yet. The question is if we want this? How frequently is this
used? This would require a means to describe the coordinate system and another attribute
on each axis which denotes in which coordinate system it acts.

Otherwise the main difference is that NeXus does not collect all the axes in one place but
keeps them at their place in the hierarchy. The attributes we added recently allow for a
full mapping. Especially, NeXus, does not single out a goniometer but stores the relevant
fields in the NXsample group. The argument being: this is stuff which positions the sample
and thus belongs into NXsample. The question here is: does NeXus need to change and
devise a NXgoniometer group?

Section 3.9, Diffrn_data_frame, Clarification
-------------------------------------------------

I fail to graps the concept even from the CBF documentation. The appears as if this
allows to group data from various detectors together. Is this right? What is this for?

Section 3.10, Some more NXdetetcor fields
--------------------------------------------

Some more fields for NXdetector in here.

Section 3.11, Diffrn_measurement
------------------------------------

See 3.8, coordinate systems and comments on NXsample and goniometers.

Section 3.12, P29, Diffrn_radiation, NXpolariser
-----------------------------------------------------

Some of the fields here do not belong into NXmonochromator but into
NXpolariser. Which needs some fields added.

What is SIEGBAHNTYPE, IUPACXRAYSMB?

Section 3.14.1, 3.14.2, P32-33, scan axis mapping,
------------------------------------------------

This is again about associating scan axis and data. See comment  on 3.7.3

Section 3.16, P36, Variant feature
------------------------------------

This is actually a feature missing in NeXus: variants. This means that there are
versions of a parameter: for example you have a theoretical beam_center you use in the
first tages, then you refine it and have another beam center and so on. There are as
of now 4 suggestions how to add this feature to NeXus:
- Use NXsubentry. Has the disadvantage to reproduce the whole tree. Also not the purpose
   of NXsubentry
- A sepcial NXvariant group. Can occur in any group and contains different values for fields
   of the parent group
- Simply add _variant to the normal NeXus name. If there multiple variants add: _variant_role
   where role becomes an indcator for the type of the variant: may be refined, from XXXplot, etc
- Ask HDFgroup if there is a HDF-5 mechanism to implement versions. Or planned.

This needs discussion.

Section 4, P38, Eiger
---------------------

Probably a typo: angular_calibration_applied is a boolena, not an array. The matching
array would be angular_calibration.

_______________________________________________
NeXus-committee mailing list
NeXus-committee at nexusformat.org
http://lists.nexusformat.org/mailman/listinfo/nexus-committee

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom