[Nexus-developers] NeXus and CIF
Mark Koennecke
Mark.Koennecke at psi.ch
Mon Feb 1 08:41:37 GMT 2010
Hi everyone,
at the ERSF HDF-5 workshop it was suggested to try a mapping from imgCIF
to NeXus. Herbert
Bernstein made some steps into this direction by writing a program to
map a CIF file into something
NeXus like. An example of this is in the attached file
mb_LP_1_001_orig_cf.hdf5. The meaning of the
fields (and some more) is explained in the attached paper
cif_img_1.3.2.dic.pdf. I the tried a mapping
from Herberts CIF file to NeXus proper. This is together with my
comments in imgcif-NeXus.map.
This is plain ASCII, viewable in any decent editor. I forwarded this
first to Herbert, mainly because I
was unsure if I got everything about imgCIF right. Below are his
comments on this.
A word of warning: there are lots of confusing ID's in imgCIF files
which relate data items with each
other. IMHO they are not really needed in NeXus files as the
relationship between data items in NeXus
is established by the hierarchy.
My personal opinions on all this:
* The axis specifications in imgCIF are superior to what we do. I think
we steal something from them
here. There are various ways to do that:
** Use it for axis documentation purposes in dictionaries and
application definitions
** Use appopriate attributes at movers to define the axis and its
action better
** Modify NXgeometry to use imgCIF schemes.
* There are other issues about how to store data: imgCIF allows to store
arbitrary byte orders, data with offsets
and correction factors etc. NeXus requires C storage order and
proper data. If we want to allow the CIF scheme
I suggest adding their keywords.
* There are some minor issues about fields to be added to base classes.
* I think Herbert is slightly wrong when he says in his comment that no
arbitrary scan can be expressed in NeXus.
I know this a bit tedious, but please have a look.
Best Regards,
Mark
------------------- Herbert Bernsteins Comments
----------------------------
Dear Mark,
Your example and comments have been very helpful. It has helped to
realize that we need to break down the steps to be taken into layers
to assure
a full interoperability between imgCIF and NeXus.
Let us begin with the handling of the coordinate frames and scans.
To follow what I am going to say, you would find it helpful to refer
to the imgCIF dictionary at:
http://www.bernstein-plus-sons.com/software/CBF/doc/cif_img_1.5.4.html
The bottom line is that the NeXus coordinate and scan specification
can be mapped into imgCIF, but the greater generality of the imgCIF
specification would be difficult to map into NeXus without adding
signficantly to the NeXus definitions. Both approaches are able to
specify scan axis step, by axis step, but the imgCIF approach
works with arbitrary assemblies of fixed and positioning axes essentially
using the fully assembled beam-line as-built engineering drawings,
and we will need a place to hold all that information.
The coordinate systems in NeXus are based on the McStas coordinate
system. In that system, the laboratory coordinate system is based on
a Z-axis point in the direction of the beam, the X-axis is
perpendicular to the beam, in the horizontal plane point left as seen
from the source, and the y-axis point upwards to complete a
right-handed system. The imgCIF laboratory coordinate system is
based on a Z-axis point from the sample into the beam, an X-axis
given by the principal axis of the goniometer and a Y-axis completing
a right-handed system. In order to avoid confusion between these
somewhat different coordinate systems, let us use the labels Xnx,
Ynx, and Znx for the NeXus laboratory coordinate system axes and
Xcif, Ycif and Zcif for the imgCIF laboratory coordinate system axes.
The NeXus approach to defining positions of components is define a
NXgeometry instance (object) for the component (with every instance
named "geometry" to allow linking under HDF rules that require the
same name for linked objects. An NXgeometry instance has an NXshape
instance, and NXtranslation instance and an NXorientation instance as
well as a description a component_index. The shape is given by
spherical, cylindrical or orthogonal coordinate system extents of
multiple objects in terms of a local set of axis after NXorientation
with the center of mass at the local origin after NXtranslate, with
NXtranslate applied before NXorientation.
An NXtranslation instance gives a displacement vector relative
either to the laboratory Xnx, Ynx, Znx coordinate system or to the
local coordinates of another object. An NXorientation object gives
the orientation of the local coordinate axes relative to the
coordinate axes on which it depends as a set of direction cosines.
The local Xnx' and Ynx' vectors are given in terms of the coordinate
system on which it depends explicitly, The third local coordinate
vector, Xnx' is implied by the assumption of a right-handed
coordinate system.
The imgCIF convention for specifying positions of entities of
interest in an experiment in terms of axes according to the rules of
the AXIS category. The default is to specify all axes in terms of
the imgCIF laboratory coordinate system. In addition, axes may be
specifies in terms of the direct lattice, cell Cartesian orthogonal
coordinates, the reciprocal lattice and an abstract Cartesian
coordinate system. The values of the tags _axis.offset[1],
_axis.offset[2], _axis.offset[3] specify the base of an axis. The
values of the tags _axis.vector[1], _axis.vector[2], _axis.vector[3]
specify the unit vector of an axis.
Thus in the static position case, the mapping from imgCIF to NeXus is
a matter of computing appropriate relative coordinate systems for
dependent axes. The mapping from NeXus to imgCIF is a matter of
computing absolute versions of coordinate frames for relative ones.
The major difference in handling coordinates arise when considering a
scan. The NeXus approach is to gather angle settings that are
changing into arrays as an attribute of the data array. The imgCIf
approach is to have multiple images, each as a separate frame, either
in separate files or all in the same file. The relationship between
scans and detailed axis setting is held in the DIFFRN_SCAN,
DIFFRN_SCAN_FRAME and DIFFRN_SCAN_FRAME_AXIS categories, allowing
completely general and detailed specification of any arbitrary
pattern of rotation and translation axis settings with any ordered
set of images. This means that, if the mechanical relationships of
the NeXus scan axes are completely specified in the NeXus file they
can be represented in an imgCIF, but there is no convenient way to
carry the full generality of the imgCIF scan axis settings in the
NeXus file without adding to the NeXus definitions.
For these reasons, we suggest the following additions to the NeXus definitions:
Import of the entire imgCIF category as a new NeXus class NXcif_axis:
<NXcif_axis name="name of the axis">
<name type="NX_CHAR">
{Descriptive name of axis}?
</name>
<depends_on type="NX_CHAR">
{name of the axis on which this axis depends, if any}?
</depends_on>
<equipment type="NX_CHAR">
"goniometer"|"detector"|"general"|"gravity"|""source"
</equipment>
<offset type="NX_FLOAT[3]">
{the base of the axis vector in the specified coordinate system}?
</offset>
<type type="NX_CHAR">
"rotation"|"translation"|"general"
</type>
<system type="NX_CHAR">
"imgCIF_laboratory"|NeXus_laboratory"|"direct"|"orthogonal"|"reciprocal"|"abstract"|
</system>
<vector type="NX_FLOAT[3]">
{the base of the axis vector in the specified coordinate system}?
</vector>
<NXcif_axis>
with similar direct mappings for the DIFFRN_SCAN, DIFFRN_SCAN_FRAME
and DIFFRN_SCAN_FRAME_AXIS categories.
I hope you will review this suggestion and give me your comments.
Regards,
Herbert
At 3:00 PM +0100 1/21/10, Mark Koennecke wrote:
>Herbert,
>
>Herbert J. Bernstein wrote:
>>Dear Mark,
>>
>> Thank you for this interesting mapping.
>>
>> In all cases a place will need to be found for every tag. When
>>you say "not needed", I assume you are referring to what is needed
>>for the integrity of the NeXus file per se, but the information
>>itself, is needed in order to be able to be able to recover the
>>entire original imgCIF file from what is placed in the NeXus tree.
>>Certainly, we can partially populate the NeXus tree and keep the
>>"real" information only in the flat CIF tables, but this just
>>postpones the problem, because for the database view we were
>>discussing with SOLEIL, _every_ one of the CIF tags will be needed.
>>
>This is a misunderstanding. I referred to ID's. From the imgcif
>publication I understood the ID's as means to connect various pieces
>of data in the file. I never understood them as database ID's (like
>primary keys?) either. The information is there: in NeXus the
>relationship of
>data is encoded through the hierarchy, through placing fields
>together in groups.
>
>> What is most critical is not the mapping from imgCIF into NeXus.
>>As I noted above, that can be dealt with by simply keeping the
>>"real" information in the flat tables. No, the critical issue is
>>mapping in the other direction -- from an arbitrary HDF5 NeXus tree
>>into appropriate tables ready for use for SQL queries.
>>
>In order to facilitate the NeXus --> CIF path I attached an example
>file in NeXus similar to the one you gave me:
>a scan on a rotation camera. This has been converted from an
>eulerian cradle diffractometer file; the structure is all right, the
>numbers in the data may not be meaningful.
>> I will take what you have done here and use it as a base for that
>>mapping and send you back a complete bi-directional proposal by the
>>end of next week.
>>
>> Regards,
>> Herbert
>>
>>P.S. This is not a matter of NeXus vs. CIF or anything vs.
>>anything, but of organizing things properly in any and all
>>available formats so that no information is lost and so that
>>information can easily and efficiently be interchanged.
>>
>I agree. This was bad wording.
>
>Mark
>
>>=====================================================
>> Herbert J. Bernstein, Professor of Computer Science
>> Dowling College, Kramer Science Center, KSC 121
>> Idle Hour Blvd, Oakdale, NY, 11769
>>
>> +1-631-244-3035
>> yaya at dowling.edu
>>=====================================================
>>
>>On Thu, 21 Jan 2010, Mark Koennecke wrote:
>>
>>>Dear Herbert Bernstein,
>>>
>>>I attached a text file which contains an attempt at an mapping between your
>>>NeXus converted imgCIF file to NeXus. I send this to you first because
>>>you may be able to spot occurrences where I misrepresent imgcif.
>>>
>>>Best Regards,
>>>
>>> Mark Koennecke
>>>
>
>
>
>
>Attachment converted: Macintosh HD:nxxrot.h5.gz ( / ) (003731A9)
--
=====================================================
Herbert J. Bernstein, Professor of Computer Science
Dowling College, Kramer Science Center, KSC 121
Idle Hour Blvd, Oakdale, NY, 11769
+1-631-244-3035
yaya at dowling.edu
=====================================================
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: imgcif-NeXus.map
URL: <http://lists.nexusformat.org/pipermail/nexus-developers/attachments/20100201/a9424156/attachment-0001.el>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mb_LP_1_001_orig_cg.hdf5
Type: application/octet-stream
Size: 9366881 bytes
Desc: not available
URL: <http://lists.nexusformat.org/pipermail/nexus-developers/attachments/20100201/a9424156/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cif_img_1.3.2.dic.pdf
Type: application/pdf
Size: 134911 bytes
Desc: not available
URL: <http://lists.nexusformat.org/pipermail/nexus-developers/attachments/20100201/a9424156/attachment-0001.pdf>
More information about the NeXus-developers
mailing list