Storing repeating structures in NeXus files

[Ray Osborn]

on 99/11/3 10:35 AM, C.M.Moreton-Smith at rl.ac.uk at
C.M.Moreton-Smith at rl.ac.uk wrote:

> I've spent some time talking to Freddie about this problem and have come up
> with a two proposals.  As Ray points out, this should be posted to the NeXus
> list but I'd rather the NAPI list had a chance to comment first before I
> post it.
> 
> The main aim here is to offer a single convention for handling repeating
> structures rather than leave it to everyone to implement them in a slightly
> different way.  In the case of ISIS files, there are several examples of
> needing this sort of structure, one NXEntry per period is a  likely one, or
> possibly a two dimensional ordering of NXentries based on periods in one
> dimension and time-regimes in the other.  Any common parameters are simply
> linked back to the first entry so the structure remains fairly efficient.
> The data group would of course be different in each entry.
> 

Unless the use of periods and time regimes has changed dramatically since I
worked at ISIS, I think they do not justify the use of group arrays.  I
would consider it overkill.

1) Periods:  The periods that Chris is referring to are very like the frames
that Mark discussed.  In both cases, my preferred solution would be to add
an extra dimension to the NXdata.   In my experience, there are usually only
a handful of periods.  For example, there may be only two periods to store
flipper-on and flipper-off data; these should definitely be stored as a
single dataset with an extra dimension of size 2.  You can then use
NXgetslab to extract each set of data.  You can have many more periods when
they are used in alignment scans, e.g. on PRISMA, but that is a fudge to
disguise the inflexibility of the standard ICP to cope with scan data.  The
data are still much better stored as a multidimensional array; they will be
much easier to plot and analyze.

2) Time regimes: When I was at ISIS, no instruments had more than one time
regime because each regime required a new crate, which was expensive.
Surely no instrument has more than a few even now.  Since each time regime
has a different set of time channel boundaries, they clearly need to be
stored in separate NXdata groups.  However, this is no different from
storing different detector banks in different groups - they can be called
whatever you like (bank1, 4m_bank, low_angle_bank etc.)  They don't
logically form an array since there is no obvious variable linking them.

I think that you may be trying to make too literal a translation of the ISIS
run file.  There are times when its structure is more complicated than it
need be because of its own design limitations, but I believe we should use
the flexibility of the NeXus structure to simplify it, not make NeXus more
complicated.  

> A smaller scale and more ISIS specific example is that of sample environment
> parameter blocks which are already stored as an array of similar
> heterogeneous structures, one for each piece of sample environment equipment
> on the CAMAC control system.  Writing these as separate arrays is
> incovenient, if only because some would have to be arrays of strings (sample
> block names).
> 

Again, I believe that we are talking about a handful (perhaps up to 20?)
sample blocks.  Actually, putting all the sample blocks into groups called
"1", "2" etc, will make it much more difficult for the user to find the one
s/he's interested in.  Surely, s/he would prefer to find groups called
"Temperature" or "Sample_changer", when browsing through the files.  Again,
I think you are trying to map the existing ISIS file structure onto a NeXus
file structure with as little modification as possible.  I would prefer it
if you looked at the current glossary, and tried to see where you would
store the information the sample blocks contain.  If there is no appropriate
location, we need to add things to the glossary.

To summarize, I have yet to be convinced that we need the complication of
group arrays.  The current NeXus design, if used properly, can provide more
efficient and elegant solutions.

Regards,
Ray
-- 
Dr Ray Osborn                Tel: +1 (630) 252-9011
Materials Science Division   Fax: +1 (630) 252-7777
Argonne National Laboratory  E-mail: ROsborn at anl.gov
Argonne, IL 60439-4845