[NeXus-committee] Concordance paper, ECM
yayahjb
yayahjb at gmail.com
Wed Apr 24 03:49:37 BST 2013
I kept going on the Dectris Eiger application definition and CIF mapping.
I broke up the application definition into one per class and renamed some
of the classes to begin with NX and avoid conflicts with existing
definitions
until we can see if we agree on extending exsiting ones. I have extracted
just the portion of the 112 page document that deals with the Eiger.
More follows later this week.
-- Herbert
On 4/22/13 7:52 AM, yayahjb wrote:
> I'll go through all Mark's comments carefully later today, but his
> last one is
> very important -- I had some typos in the Eiger application definition
> in addition to the one Mark found. Here is an update, hopefully with
> fewer errors. Further proofreading would be much appreciated.
>
>
> NXdectris_eiger (application definition, version 0.1)
> (overlays NXentry)
> entry:NXentry
> data_000001:NXDATA
> data:NXINT[np_000001,number of x pixels,number of y pixels]
> @image_nr_low
> @image_nr_high
> data_000002:NXDATA
> data:NXINT[np_000001,number of x pixels,number of y pixels]
> @image_nr_low
> @image_nr_high
> ...
> data_nnnnnn:NXDATA
> data:NXINT[np_nnnnnn,number of x pixels,number of y pixels]
> @image_nr_low
> @image_nr_high
> instrument:NXinstrument
> detector:NXdetector
> acquisition_mode:NX_CHAR
> angular_calibration_applied:NX_BOOLEAN
> beam_center_x:NX_FLOAT
> @units
> beam_center_y:NX_FLOAT
> @units
> bit_depth_readout:NX_UINT
> count_time:NX_FLOAT[np]
> @units
> countrate_correction_applied:NX_BOOLEAN
> description:NX_CHAR
> detector_number:NX_CHAR
> detectorExtended:DetectorExtended
> detector_distance:NXFLOAT
> @units
> rotation_angle_step:NX_FLOAT[np]
> @units
> wavelength:NX_FLOAT
> @units
> detectorSpecific:DetectorSpecific
> countrate_correction_bunch_mode:NX_CHAR
> countrate_correction_count_cutoff:NX_UINT
> countrate_correction_lookup_table:NX_FLOAT[1000000]
> data_collection_date:NX_CHAR
> detectorModule_000:DetectorModule
> dac_names:NX_CHAR[6]
> dac_values:NX_UINT[7]
> data_origin:NX_UINT[2]
> data_size:NX_UINT[2]
> detectorChip_00:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> detectorChip_01:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> ...
> detectorChip_15:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> fast_pixel_direction:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> firmware_version:NX_CHAR
> module_offset:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> nbits:NX_UINT
> nchips:NX_UINT
> readout_frequency:NX_FLOAT
> @units
> region_of_interest:NX_UINT[4]
> slow_pixel_direction:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> x_pixels_in_module:NX_UINT
> y_pixels_in_module:NX_UINT
> detectorModule_001:DetectorModule
> dac_names:NX_CHAR[6]
> dac_values:NX_UINT[7]
> data_origin:NX_UINT[2]
> data_size:NX_UINT[2]
> detectorChip_00:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> detectorChip_01:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> ...
> detectorChip_15:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> fast_pixel_direction:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> firmware_version:NX_CHAR
> module_offset:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> nbits:NX_UINT
> nchips:NX_UINT
> readout_frequency:NX_FLOAT
> @units
> region_of_interest:NX_UINT[4]
> slow_pixel_direction:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> x_pixels_in_module:NX_UINT
> y_pixels_in_module:NX_UINT
> ...
> detectorModule_059:DetectorModule
> dac_names:NX_CHAR[6]
> dac_values:NX_UINT[7]
> data_origin:NX_UINT[2]
> data_size:NX_UINT[2]
> detectorChip_00:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> detectorChip_01:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> ...
> detectorChip_15:DetectorChip
> chip_type:NX_CHAR
> x_pixels_in_chip:NX_UINT
> x_position:NX_UINT
> y_pixels_in_chip:NX_UINT
> y_position:NX_UINT
> fast_pixel_direction:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> firmware_version:NX_CHAR
> module_offset:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> nbits:NX_UINT
> nchips:NX_UINT
> readout_frequency:NX_FLOAT
> @units
> region_of_interest:NX_UINT[4]
> slow_pixel_direction:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> x_pixels_in_module:NX_UINT
> y_pixels_in_module:NX_UINT
> detector_origin:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> detector_origin_rotated:NX_FLOAT64
> @depends_on
> @transformation
> @units
> @vector
> flat field:NX_FLOAT[number of x pixels,number of y pixels]
> imgType:NX_UINT
> mode_register:NX_UINT
> nimages:NX_UINT
> number_of_excluded_pixels:NX_UINT
> photon_energy:NX_FLOAT
> pixel_mask:NX_FLOAT[number of x pixels,number of y pixels]
> readout_mode:NX_CHAR
> seriedId:NX_ULONG
> software_version:NX_CHAR
> sub_image_exposure_time:NX_FLOAT
> summation_mode:NX_CHAR
> summation_nimages:NX_UINT
> trigger_mode:NX_CHAR
> x_pixels_in_detector:NX_UINT
> y_pixels_in_detector:NX_UINT
> detector_number:NX_CHAR
> detector_readout_time:NX_FLOAT[np]
> @units
> efficiency_correction_applied:NX_BOOL
> flatfield_correction_applied:NX_BOOL
> frame_time:NX_FLOAT[np]
> @units
> gain_setting:NX_CHAR
> number_of_cycles:NX_UINT
> pixel_mask_applied:NX_BOOL
> sensor_material:NX_STRING
> sensor_thickness:NX_FLOAT
> @units
> threshold_energy:NX_FLOAT
> @units
> virtual_pixel_correction_applied:NX_BOOL
> x_pixel_size:NX_FLOAT
> @units
> y_pixel_size:NX_FLOAT
> @units
>
>
>
> On 4/22/13 6:43 AM, Tobias.Richter at diamond.ac.uk wrote:
>> Hi all,
>>
>> Just to reiterate what Herbert has written in the document: This
>> draft is a work in progress. In particular this version is one that I
>> didn't have time to comment on.
>>
>> The idea behind this whole operation is to make a minimum of changes
>> in either format. If we need to preserve information that currently
>> cannot be expressed we add a place for it. Everything else should go
>> into existing slots.
>> In rare cases were the existing slots are not suitable, we may need
>> to make amends. But I would say we clearly haven exhausted our
>> options yet. That's why we ask for comments.
>>
>> In line with the NeXus philosophy there a number (potentially
>> optional) fields and attributes may be added to aid a faithful
>> roundtrip conversion between NeXus and CBF. There is a fair number
>> of fields from that category and where we aren't mapping correctly
>> yet, it would probably we helpful to identify which is which.
>>
>> Just one comment on a few points raised by Mark (leaving the major
>> share for Herbert):
>>
>> Section 3.2, P 10, Admin Feature Request
>> ---------------------------------------------------------
>> We already have experiment_identifier and entry_identifier defined.
>> With entry = scan (more or less) we would only need a
>> diffraction_identifier (to keep the same naming convention), which in
>> my understanding would be a per-frame array. Right?
>>
>> Section 3.16, P36, Variant feature
>> ----------------------------------------------
>> NXsubentry addresses a different problem. You might also want to
>> record a variant of something that is in a subentry.
>> My current idea is that of a NXvariant group that inherits from
>> NXentry and also live at the root level. It would reference the
>> NXentry or other NXvariant it operates on (in e.g. an attribute) and
>> would only carry the item at are changed and the required structure
>> to hold them.
>> An alternative would be to link the unchanged entries. It keeps
>> reading the variant simple in software but that hides the purpose of
>> that feature and makes the changes less accessible to humans.
>>
>>
>> I am not sure I see much scope for discussing this with the CIF
>> committee. I am not even sure they have seen this document yet.
>>
>> Interest for doing anything in August was low in the Doodle poll. We
>> have some discussions around the ECM meeting if people change their
>> mind. For the moment I do have Jonathan devoted to the project. We
>> have to keep making progress.
>>
>> Regards,
>>
>> Tobias
>>
>> ________________________________________
>> From: nexus-committee-bounces at nexusformat.org
>> [nexus-committee-bounces at nexusformat.org] on behalf of Mark Koennecke
>> [Mark.Koennecke at psi.ch]
>> Sent: 22 April 2013 10:14
>> To: Discussions of the NeXus Advisory Committee
>> Subject: [NeXus-committee] Concordance paper, ECM
>>
>> Hi everyone,
>>
>> There are two things which we need to be decide upon rather quickly
>> in order to allow planning to get done.
>>
>> The first is that it occurred to me that merging NeXus and CIF might
>> mean that we need to understand their dictionary technology. So may be
>> a few of us should ask to join the CIF workshop, August 23-24. I myself
>> would be interested. Anyone else?
>>
>> The second is that Herberts concordance paper raised a number of issues.
>> Thus I wonder if we should not have a 1 day code camp/meeting before
>> meeting with the CIF people to sort this out. Anyone opinions on this?
>>
>> I also took a little time to look through the concordance paper. Find my
>> comments attached below.
>>
>>
>> Best Regards,
>>
>> Mark Koennecke
>>
>> Notes on Herberts Concordance Summary: 4/2013
>> ================================================
>>
>> Herberts paper raises a few issues and asks for some new features in
>> NeXus.
>> This is my humble attempt to summarize and comment the issues.
>>
>> Section 2, Page 2
>> --------------------
>>
>> I was a little confused: As I see it this comes down to:
>> CBF uses tables, NeXus hierarchical groups. The basic mapping is
>> a CBF table to a NeXus group. CBF table columns map to NeXus fields.
>> In many cases a CBF table has only one row. If there are multiple rows
>> in CBF for example, NR,then in NeXus the fields will be
>> arrays of length NR.
>>
>> Page 3: During the NeXus Telco this was resolved: different scans map to
>> different NXentries/ files.
>>
>>
>> Section 3.2, P 10, Admin Feature Request
>> ------------------------------------------
>>
>> This is a feature request: especially in PX there is a deeper level
>> hierarchy of experiments: there is the study, there are multiple
>> crystals
>> (of the same protein) and there are multiple scans on those crystals.
>> In order
>> not to loose this hierarchy it is needed to include this administrative
>> information in each file. Herbert suggest 3 additional fields or
>> attributes:
>> scan_id, diffrn_id, entry_id at NXentry level. I personally prefer
>> fields.
>>
>>
>> Section 3.3, P10, Multiple detectors
>> -----------------------------------------
>>
>> NeXus deals with multiple detectors or ROI's by storing data in
>> separate NXdetector groups. Which makes a lot of sense. Q: Why does this
>> fall over?
>>
>> Section 3.6, P12-13, NXdetector fields
>> -------------------------------------
>>
>> This asks for some new fields in NXdetector.
>>
>>
>> Section 3.7.1, P14, Array_Structure_List
>> ------------------------------------------
>>
>> I compared this with the CBF description. This is about array order
>> and dimensions etc.
>> IMHO, NeXus covers this by the HDF-5 dimensions and the offset and
>> stride attributes.
>>
>>
>> Section 3.7.2, P15, array_structure_list_section, ROI feature missing
>> -------------------------------------------------------------------
>>
>> Again looking at the CBF documentation this seems to declare a region
>> of interest into
>> a larger array. NeXus does not have this per se. Currently I fail to
>> understand how this
>> is used: will there be another data array? Or does this mean: I have
>> this big fat array,
>> but just use a smaller part of it?
>>
>>
>> Section 3.7.3, P 15-16, array_structure_list_axis, CBF-NeXus difference
>> -------------------------------------------------------------------------
>>
>>
>> This is about associating scan axis with the data array. CBF does
>> this by stating
>> a lot of parameters: starts, increments etc. for each axis. Or
>> appears to be doing
>> this. I might have misread the CBF docs. In NeXus this is solved by
>> storing actual
>> positions along the scan in arrays and the association is done by the
>> scan rules,
>> most notably the population of NXdata in the scan case. This actually
>> may require some
>> discussion: IMHO, storing arrays is better as it allows for the
>> possibility of
>> mispositionings.
>>
>>
>> Section 3.8, Axis, Coordinate System Features
>> --------------------------------------------------
>>
>> Here is a major difference: CBF supports multiple coordinate systems.
>> Correspondingly CBF has a way to document the coordinate system used
>> by defining the
>> new coordinate system in terms of the standard laboratory system and
>> by defining the
>> directions of gravity and the beam.
>>
>> NeXus does not do this yet. The question is if we want this? How
>> frequently is this
>> used? This would require a means to describe the coordinate system
>> and another attribute
>> on each axis which denotes in which coordinate system it acts.
>>
>> Otherwise the main difference is that NeXus does not collect all the
>> axes in one place but
>> keeps them at their place in the hierarchy. The attributes we added
>> recently allow for a
>> full mapping. Especially, NeXus, does not single out a goniometer but
>> stores the relevant
>> fields in the NXsample group. The argument being: this is stuff which
>> positions the sample
>> and thus belongs into NXsample. The question here is: does NeXus need
>> to change and
>> devise a NXgoniometer group?
>>
>>
>> Section 3.9, Diffrn_data_frame, Clarification
>> -------------------------------------------------
>>
>> I fail to graps the concept even from the CBF documentation. The
>> appears as if this
>> allows to group data from various detectors together. Is this right?
>> What is this for?
>>
>>
>> Section 3.10, Some more NXdetetcor fields
>> --------------------------------------------
>>
>> Some more fields for NXdetector in here.
>>
>>
>> Section 3.11, Diffrn_measurement
>> ------------------------------------
>>
>> See 3.8, coordinate systems and comments on NXsample and goniometers.
>>
>>
>> Section 3.12, P29, Diffrn_radiation, NXpolariser
>> -----------------------------------------------------
>>
>> Some of the fields here do not belong into NXmonochromator but into
>> NXpolariser. Which needs some fields added.
>>
>> What is SIEGBAHNTYPE, IUPACXRAYSMB?
>>
>>
>> Section 3.14.1, 3.14.2, P32-33, scan axis mapping,
>> ------------------------------------------------
>>
>> This is again about associating scan axis and data. See comment on
>> 3.7.3
>>
>>
>> Section 3.16, P36, Variant feature
>> ------------------------------------
>>
>> This is actually a feature missing in NeXus: variants. This means
>> that there are
>> versions of a parameter: for example you have a theoretical
>> beam_center you use in the
>> first tages, then you refine it and have another beam center and so
>> on. There are as
>> of now 4 suggestions how to add this feature to NeXus:
>> - Use NXsubentry. Has the disadvantage to reproduce the whole tree.
>> Also not the purpose
>> of NXsubentry
>> - A sepcial NXvariant group. Can occur in any group and contains
>> different values for fields
>> of the parent group
>> - Simply add _variant to the normal NeXus name. If there multiple
>> variants add: _variant_role
>> where role becomes an indcator for the type of the variant: may
>> be refined, from XXXplot, etc
>> - Ask HDFgroup if there is a HDF-5 mechanism to implement versions.
>> Or planned.
>>
>> This needs discussion.
>>
>>
>> Section 4, P38, Eiger
>> ---------------------
>>
>> Probably a typo: angular_calibration_applied is a boolena, not an
>> array. The matching
>> array would be angular_calibration.
>>
>>
>>
>>
>> _______________________________________________
>> NeXus-committee mailing list
>> NeXus-committee at nexusformat.org
>> http://lists.nexusformat.org/mailman/listinfo/nexus-committee
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DECTRIS_Draft_Appdef.pdf
Type: application/pdf
Size: 95076 bytes
Desc: not available
URL: <http://lists.nexusformat.org/pipermail/nexus-committee/attachments/20130423/adbb11f1/attachment.pdf>
More information about the NeXus-committee
mailing list