[NeXus-committee] DECTRIS example summary

[Pete Jemian]

Armando's point is right (or on the right track, at least).

DECTRIS (and others) need to shovel large batches of data (image frames) 
into HDF5 files as fast as possible.  For them, this means a very 
shallow structure - no subgroups - just data in the root directory.  For 
multiple access reasons (pointed out earlier today), each new batch goes 
into a new file.

A master NeXus file needs to identify all the files and parts so they 
can be reassembled into a sensible and complete set of data.  Since, in 
the DECTRIS plan, they are representing slices of multi-dimensional data 
in different HDF5 datasets, the resulting NeXus master file needs to be 
able to describe the final structure and reference the locations of all 
the parts.  The parts can be found using external links.  We don't have 
a way to describe how the parts fit into the multi-dimensional data. 
That's needed.

I'm not sure we decided (maybe because I was not in the room at the 
time) to move the signal attribute from the dataset to the parent NXdata 
group.  Even if we did, for sure, the manual does not say that.  Yet. 
I'm convinced NeXus should do this.  The DECTRIS data shows why we would 
be so motivated.

Pete

On 1/14/2013 2:44 PM, V. Armando Sole wrote:
> Hi Eugene,
>
> That still does not make transparent the link mechanism (different
> paths) and does not solve the problem that in fact we are segmenting
> what in other case would be a single dataset as Ben pointed out.
>
> Following that approach, the main thing to be made to the supplied file is:
>
> - Make sure all the links are in the same NXdata group in the master file.
> - Add a *new* attribute to that NXdata group in order to let us know the
> "signal" inside is build up of a set of links.
>
> Anybody is free to come with other suggestion. This one has the
> advantage that DECTRIS has almost nothing to do (make sure the links
> -not the targets- are inside an NXdata group) and everything can be
> solved at the master file level (adding attributes to the NXdata group
> in line with the new approach of defining the plots).
>
> Armando
>
> On 14.01.2013 21:24, Wintersberger, Eugen wrote:
>> Hi all
>>   Ok - if I got the entire discussion right the problem is to access
>> metadata for the different blocks as links are entirely transparent in
>> HDF5. In order to increase confusion let me make another proposal
>> (though I am not sure if someone has already made a suggestion like this
>> - I somehow lost track)
>>
>> master file:
>> -----------
>> entry:NXentry/instrument:NXinstrument/detector_1
>> ....
>> entry:NXentry/instrument:NXinstrument/detector_n
>> entry:NXentry/data:NXdata/data_1 (link to data in detector_1)
>> ...
>> entry:NXentry/data:NXdata/data_n (link to data in detector_n)
>>
>> detector_i is a link to the detector group in the file for the i-th
>> image block
>>
>> image block file:
>> ----------------
>> entry:NXentry/instrument:NXinstrument/detector:NXdetector/data
>> first_image_nr
>> last_image_nr
>> entry:NXentry/data:NXdata/data (link to data in detector group)
>>
>> To my opinion this approach has several advantages:
>> 1.) because the detector group is linked to the master file all the
>>     detector related metadata can be accessed easily
>> 2.) as the entire detector group is available the different image
>>     blocks can be of different pixels size, geometry, or what ever
>> 3.) I have it when detector data is not stored in an NXdetector group
>>     along with the metadata.
>>
>> best regards
>>   Eugen
>>
>> On Mon, 2013-01-14 at 14:38 +0100, Mark Koennecke wrote:
>>> Hi,
>>>
>>> from your feedback and my own study I find that there are the following
>>> problems
>>> with the example file as is:
>>>
>>> - Structure is wrong regarding linked data sets. Should be either:
>>>
>>>    * entry:NXentry/data_00001:NXdata/data (Preferred ....)
>>>    * entry:NXData/data:NXdata/data_0001
>>>
>>> - On data, signal=1 attribute is missing
>>> - On external link, NeXus attribute
>>> NAPIMOUNT=nxfile://th02c_ps02_1_data_000001.h5#data
>>>    missing
>>> - In detector, flatfield, efficiency data is missing. This is not
>>> critical,
>>>    I just wonder.
>>>
>>> Am I right? Is this what I tell DECTRIS tomorrow? Or do I miss
>>> something?
>>>
>>> Also, what is the opinion, do we need another Telco on this one?
>>>
>>> Regards,
>>>
>>>               Mark
>>>
>>>
>>> _______________________________________________
>>> NeXus-committee mailing list
>>> NeXus-committee at nexusformat.org
>>> http://lists.nexusformat.org/mailman/listinfo/nexus-committee
>
> _______________________________________________
> NeXus-committee mailing list
> NeXus-committee at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus-committee