[NeXus-committee] Concordance paper, ECM
yayahjb
yayahjb at gmail.com
Mon Apr 22 12:52:55 BST 2013
I'll go through all Mark's comments carefully later today, but his last
one is
very important -- I had some typos in the Eiger application definition
in addition to the one Mark found. Here is an update, hopefully with
fewer errors. Further proofreading would be much appreciated.
NXdectris_eiger (application definition, version 0.1)
(overlays NXentry)
entry:NXentry
data_000001:NXDATA
data:NXINT[np_000001,number of x pixels,number of y pixels]
@image_nr_low
@image_nr_high
data_000002:NXDATA
data:NXINT[np_000001,number of x pixels,number of y pixels]
@image_nr_low
@image_nr_high
...
data_nnnnnn:NXDATA
data:NXINT[np_nnnnnn,number of x pixels,number of y pixels]
@image_nr_low
@image_nr_high
instrument:NXinstrument
detector:NXdetector
acquisition_mode:NX_CHAR
angular_calibration_applied:NX_BOOLEAN
beam_center_x:NX_FLOAT
@units
beam_center_y:NX_FLOAT
@units
bit_depth_readout:NX_UINT
count_time:NX_FLOAT[np]
@units
countrate_correction_applied:NX_BOOLEAN
description:NX_CHAR
detector_number:NX_CHAR
detectorExtended:DetectorExtended
detector_distance:NXFLOAT
@units
rotation_angle_step:NX_FLOAT[np]
@units
wavelength:NX_FLOAT
@units
detectorSpecific:DetectorSpecific
countrate_correction_bunch_mode:NX_CHAR
countrate_correction_count_cutoff:NX_UINT
countrate_correction_lookup_table:NX_FLOAT[1000000]
data_collection_date:NX_CHAR
detectorModule_000:DetectorModule
dac_names:NX_CHAR[6]
dac_values:NX_UINT[7]
data_origin:NX_UINT[2]
data_size:NX_UINT[2]
detectorChip_00:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
detectorChip_01:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
...
detectorChip_15:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
fast_pixel_direction:NX_FLOAT64
@depends_on
@transformation
@units
@vector
firmware_version:NX_CHAR
module_offset:NX_FLOAT64
@depends_on
@transformation
@units
@vector
nbits:NX_UINT
nchips:NX_UINT
readout_frequency:NX_FLOAT
@units
region_of_interest:NX_UINT[4]
slow_pixel_direction:NX_FLOAT64
@depends_on
@transformation
@units
@vector
x_pixels_in_module:NX_UINT
y_pixels_in_module:NX_UINT
detectorModule_001:DetectorModule
dac_names:NX_CHAR[6]
dac_values:NX_UINT[7]
data_origin:NX_UINT[2]
data_size:NX_UINT[2]
detectorChip_00:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
detectorChip_01:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
...
detectorChip_15:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
fast_pixel_direction:NX_FLOAT64
@depends_on
@transformation
@units
@vector
firmware_version:NX_CHAR
module_offset:NX_FLOAT64
@depends_on
@transformation
@units
@vector
nbits:NX_UINT
nchips:NX_UINT
readout_frequency:NX_FLOAT
@units
region_of_interest:NX_UINT[4]
slow_pixel_direction:NX_FLOAT64
@depends_on
@transformation
@units
@vector
x_pixels_in_module:NX_UINT
y_pixels_in_module:NX_UINT
...
detectorModule_059:DetectorModule
dac_names:NX_CHAR[6]
dac_values:NX_UINT[7]
data_origin:NX_UINT[2]
data_size:NX_UINT[2]
detectorChip_00:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
detectorChip_01:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
...
detectorChip_15:DetectorChip
chip_type:NX_CHAR
x_pixels_in_chip:NX_UINT
x_position:NX_UINT
y_pixels_in_chip:NX_UINT
y_position:NX_UINT
fast_pixel_direction:NX_FLOAT64
@depends_on
@transformation
@units
@vector
firmware_version:NX_CHAR
module_offset:NX_FLOAT64
@depends_on
@transformation
@units
@vector
nbits:NX_UINT
nchips:NX_UINT
readout_frequency:NX_FLOAT
@units
region_of_interest:NX_UINT[4]
slow_pixel_direction:NX_FLOAT64
@depends_on
@transformation
@units
@vector
x_pixels_in_module:NX_UINT
y_pixels_in_module:NX_UINT
detector_origin:NX_FLOAT64
@depends_on
@transformation
@units
@vector
detector_origin_rotated:NX_FLOAT64
@depends_on
@transformation
@units
@vector
flat field:NX_FLOAT[number of x pixels,number of y pixels]
imgType:NX_UINT
mode_register:NX_UINT
nimages:NX_UINT
number_of_excluded_pixels:NX_UINT
photon_energy:NX_FLOAT
pixel_mask:NX_FLOAT[number of x pixels,number of y pixels]
readout_mode:NX_CHAR
seriedId:NX_ULONG
software_version:NX_CHAR
sub_image_exposure_time:NX_FLOAT
summation_mode:NX_CHAR
summation_nimages:NX_UINT
trigger_mode:NX_CHAR
x_pixels_in_detector:NX_UINT
y_pixels_in_detector:NX_UINT
detector_number:NX_CHAR
detector_readout_time:NX_FLOAT[np]
@units
efficiency_correction_applied:NX_BOOL
flatfield_correction_applied:NX_BOOL
frame_time:NX_FLOAT[np]
@units
gain_setting:NX_CHAR
number_of_cycles:NX_UINT
pixel_mask_applied:NX_BOOL
sensor_material:NX_STRING
sensor_thickness:NX_FLOAT
@units
threshold_energy:NX_FLOAT
@units
virtual_pixel_correction_applied:NX_BOOL
x_pixel_size:NX_FLOAT
@units
y_pixel_size:NX_FLOAT
@units
On 4/22/13 6:43 AM, Tobias.Richter at diamond.ac.uk wrote:
> Hi all,
>
> Just to reiterate what Herbert has written in the document: This draft is a work in progress. In particular this version is one that I didn't have time to comment on.
>
> The idea behind this whole operation is to make a minimum of changes in either format. If we need to preserve information that currently cannot be expressed we add a place for it. Everything else should go into existing slots.
> In rare cases were the existing slots are not suitable, we may need to make amends. But I would say we clearly haven exhausted our options yet. That's why we ask for comments.
>
> In line with the NeXus philosophy there a number (potentially optional) fields and attributes may be added to aid a faithful roundtrip conversion between NeXus and CBF. There is a fair number of fields from that category and where we aren't mapping correctly yet, it would probably we helpful to identify which is which.
>
> Just one comment on a few points raised by Mark (leaving the major share for Herbert):
>
> Section 3.2, P 10, Admin Feature Request
> ---------------------------------------------------------
> We already have experiment_identifier and entry_identifier defined. With entry = scan (more or less) we would only need a diffraction_identifier (to keep the same naming convention), which in my understanding would be a per-frame array. Right?
>
> Section 3.16, P36, Variant feature
> ----------------------------------------------
> NXsubentry addresses a different problem. You might also want to record a variant of something that is in a subentry.
> My current idea is that of a NXvariant group that inherits from NXentry and also live at the root level. It would reference the NXentry or other NXvariant it operates on (in e.g. an attribute) and would only carry the item at are changed and the required structure to hold them.
> An alternative would be to link the unchanged entries. It keeps reading the variant simple in software but that hides the purpose of that feature and makes the changes less accessible to humans.
>
>
> I am not sure I see much scope for discussing this with the CIF committee. I am not even sure they have seen this document yet.
>
> Interest for doing anything in August was low in the Doodle poll. We have some discussions around the ECM meeting if people change their mind. For the moment I do have Jonathan devoted to the project. We have to keep making progress.
>
> Regards,
>
> Tobias
>
> ________________________________________
> From: nexus-committee-bounces at nexusformat.org [nexus-committee-bounces at nexusformat.org] on behalf of Mark Koennecke [Mark.Koennecke at psi.ch]
> Sent: 22 April 2013 10:14
> To: Discussions of the NeXus Advisory Committee
> Subject: [NeXus-committee] Concordance paper, ECM
>
> Hi everyone,
>
> There are two things which we need to be decide upon rather quickly
> in order to allow planning to get done.
>
> The first is that it occurred to me that merging NeXus and CIF might
> mean that we need to understand their dictionary technology. So may be
> a few of us should ask to join the CIF workshop, August 23-24. I myself
> would be interested. Anyone else?
>
> The second is that Herberts concordance paper raised a number of issues.
> Thus I wonder if we should not have a 1 day code camp/meeting before
> meeting with the CIF people to sort this out. Anyone opinions on this?
>
> I also took a little time to look through the concordance paper. Find my
> comments attached below.
>
>
> Best Regards,
>
> Mark Koennecke
>
> Notes on Herberts Concordance Summary: 4/2013
> ================================================
>
> Herberts paper raises a few issues and asks for some new features in NeXus.
> This is my humble attempt to summarize and comment the issues.
>
> Section 2, Page 2
> --------------------
>
> I was a little confused: As I see it this comes down to:
> CBF uses tables, NeXus hierarchical groups. The basic mapping is
> a CBF table to a NeXus group. CBF table columns map to NeXus fields.
> In many cases a CBF table has only one row. If there are multiple rows
> in CBF for example, NR,then in NeXus the fields will be
> arrays of length NR.
>
> Page 3: During the NeXus Telco this was resolved: different scans map to
> different NXentries/ files.
>
>
> Section 3.2, P 10, Admin Feature Request
> ------------------------------------------
>
> This is a feature request: especially in PX there is a deeper level
> hierarchy of experiments: there is the study, there are multiple crystals
> (of the same protein) and there are multiple scans on those crystals. In order
> not to loose this hierarchy it is needed to include this administrative
> information in each file. Herbert suggest 3 additional fields or attributes:
> scan_id, diffrn_id, entry_id at NXentry level. I personally prefer fields.
>
>
> Section 3.3, P10, Multiple detectors
> -----------------------------------------
>
> NeXus deals with multiple detectors or ROI's by storing data in
> separate NXdetector groups. Which makes a lot of sense. Q: Why does this
> fall over?
>
> Section 3.6, P12-13, NXdetector fields
> -------------------------------------
>
> This asks for some new fields in NXdetector.
>
>
> Section 3.7.1, P14, Array_Structure_List
> ------------------------------------------
>
> I compared this with the CBF description. This is about array order and dimensions etc.
> IMHO, NeXus covers this by the HDF-5 dimensions and the offset and stride attributes.
>
>
> Section 3.7.2, P15, array_structure_list_section, ROI feature missing
> -------------------------------------------------------------------
>
> Again looking at the CBF documentation this seems to declare a region of interest into
> a larger array. NeXus does not have this per se. Currently I fail to understand how this
> is used: will there be another data array? Or does this mean: I have this big fat array,
> but just use a smaller part of it?
>
>
> Section 3.7.3, P 15-16, array_structure_list_axis, CBF-NeXus difference
> -------------------------------------------------------------------------
>
> This is about associating scan axis with the data array. CBF does this by stating
> a lot of parameters: starts, increments etc. for each axis. Or appears to be doing
> this. I might have misread the CBF docs. In NeXus this is solved by storing actual
> positions along the scan in arrays and the association is done by the scan rules,
> most notably the population of NXdata in the scan case. This actually may require some
> discussion: IMHO, storing arrays is better as it allows for the possibility of
> mispositionings.
>
>
> Section 3.8, Axis, Coordinate System Features
> --------------------------------------------------
>
> Here is a major difference: CBF supports multiple coordinate systems.
> Correspondingly CBF has a way to document the coordinate system used by defining the
> new coordinate system in terms of the standard laboratory system and by defining the
> directions of gravity and the beam.
>
> NeXus does not do this yet. The question is if we want this? How frequently is this
> used? This would require a means to describe the coordinate system and another attribute
> on each axis which denotes in which coordinate system it acts.
>
> Otherwise the main difference is that NeXus does not collect all the axes in one place but
> keeps them at their place in the hierarchy. The attributes we added recently allow for a
> full mapping. Especially, NeXus, does not single out a goniometer but stores the relevant
> fields in the NXsample group. The argument being: this is stuff which positions the sample
> and thus belongs into NXsample. The question here is: does NeXus need to change and
> devise a NXgoniometer group?
>
>
> Section 3.9, Diffrn_data_frame, Clarification
> -------------------------------------------------
>
> I fail to graps the concept even from the CBF documentation. The appears as if this
> allows to group data from various detectors together. Is this right? What is this for?
>
>
> Section 3.10, Some more NXdetetcor fields
> --------------------------------------------
>
> Some more fields for NXdetector in here.
>
>
> Section 3.11, Diffrn_measurement
> ------------------------------------
>
> See 3.8, coordinate systems and comments on NXsample and goniometers.
>
>
> Section 3.12, P29, Diffrn_radiation, NXpolariser
> -----------------------------------------------------
>
> Some of the fields here do not belong into NXmonochromator but into
> NXpolariser. Which needs some fields added.
>
> What is SIEGBAHNTYPE, IUPACXRAYSMB?
>
>
> Section 3.14.1, 3.14.2, P32-33, scan axis mapping,
> ------------------------------------------------
>
> This is again about associating scan axis and data. See comment on 3.7.3
>
>
> Section 3.16, P36, Variant feature
> ------------------------------------
>
> This is actually a feature missing in NeXus: variants. This means that there are
> versions of a parameter: for example you have a theoretical beam_center you use in the
> first tages, then you refine it and have another beam center and so on. There are as
> of now 4 suggestions how to add this feature to NeXus:
> - Use NXsubentry. Has the disadvantage to reproduce the whole tree. Also not the purpose
> of NXsubentry
> - A sepcial NXvariant group. Can occur in any group and contains different values for fields
> of the parent group
> - Simply add _variant to the normal NeXus name. If there multiple variants add: _variant_role
> where role becomes an indcator for the type of the variant: may be refined, from XXXplot, etc
> - Ask HDFgroup if there is a HDF-5 mechanism to implement versions. Or planned.
>
> This needs discussion.
>
>
> Section 4, P38, Eiger
> ---------------------
>
> Probably a typo: angular_calibration_applied is a boolena, not an array. The matching
> array would be angular_calibration.
>
>
>
>
> _______________________________________________
> NeXus-committee mailing list
> NeXus-committee at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus-committee
>
>
More information about the NeXus-committee
mailing list