[Nexus] Reading generic h5 files.
Peterson, Peter F.
petersonpf at ornl.gov
Wed Feb 21 20:36:49 GMT 2007
Brian,
Not to push one direction or the other, but I've done something similar
with xml-based NeXus and java (don't ask why). In that case the code I
generated mimicked a particular version of the NeXus api with a
particular version of mxml, including the file attributes. Since the
files are intended to be uninteresting as far as napi features go, this
worked well. The key to it all was to go through my own abstraction
layer of the file I/O which is on top of the normal java xml libraries.
I confirmed it all worked by pushing the resulting files through some of
the standard NeXus tools.
P^2
-----Original Message-----
From: nexus-bounces at nexusformat.org
[mailto:nexus-bounces at nexusformat.org] On Behalf Of tieman
Sent: Wednesday, February 21, 2007 2:58 PM
To: Ray Osborn
Cc: nexus at nexusformat.org
Subject: Re: [Nexus] Reading generic h5 files.
Freddie, Ray,
I figured the issue was something like this. For HDF4, I do create a
"type"--if it's a group that is defined in Nexus, I try and use the NX*
type (we don't have many of these), for the rest, I make something up
that would make sense to someone who might read it. When reading the
data back in, for the most part, I just pass the type along (the actual
value is quarried from the file) with the name and everyone seams to be
happy.
For HDF5, the missing NX_class attribute signals errors which ultimately
cause my code to fail. I agree that modifying the NAPI to be more
forgiving may cause problems elsewhere and maybe we can fix this in the
Matlab code. Is it really as simple as adding an "NX_class" attribute
to all groups? I did an h5dump on a file written by NAPI and I do not
see a "NX_class" attribute in the top level group "/" but I do see it in
all the other groups. Do I need to have a "Nexus_version" attribute in
"/"? Do any of the fields have special attributes that are needed?
Matlab bindings may or may not be of use to us. As you probably well
know, we only pretend to write Nexus files at the APS ;) I've thought
many times about dropping the pretense and modifying my NAPI wrapper
code to call HDF directly to avoid this confusion, but I've got a lot of
code that relies on NAPI to read/write HDF files. Nexus as an API has
served me very well over the years! If we can get Matlab to fake what
NAPI needs, we can continue to get by for now...at least until someone
with more clout than I snaps their fingers and declares the APS WILL use
Nexus as a file format...
Thanks!
Brian
Ray Osborn wrote:
> On 2007/02/21 11:53, "Akeroyd, FA (Freddie)" <F.A.Akeroyd at rl.ac.uk>
wrote:
>
>
>> Brian,
>>
>> In HDF4 objects had the concept of a "class" and this could be used
by
>> the NeXus API to store the data type; a separate group attributes was
>> not needed. In HDF5 I believe there was no such built in facility and
so
>> an "NX_class" group attribute had to be created and used instead.
>>
>
> Brian,
> The original idea with the NeXus API was that it would only be used to
read
> and write NeXus-standard-conforming files, which required that each
group
> had a name and a class associated with it. The class needed to have
the
> prefix "NX", so NAPI was only set up to read such groups. The rest
would be
> ignored - they could be there, but the NAPI did not know how to handle
them.
> HDF4 had a class automatically associated with each group, which was
> simulated in HDF5 by adding an NX_class attribute.
>
> I guess the API could be rewritten to be a bit more forgiving when
reading a
> non-standard file, but it might cause problems. Mark would be best
> qualified to comment on that. Probably the simplest thing is to
output the
> Matlab data with a script that adds the NX_class group attribute,
making
> sure it started with "NX" to be safe.
>
> I presume that the old HDF4 files worked because HDF4 automatically
gave
> each group a class. When NAPI looked for it, it found it - the HDF4
version
> doesn't seem to care whether the class has the right prefix or not.
> However, the HDF5 NAPI looks for the NX_class attribute, and complains
if
> it's not there. When you read the HDF4 Matlab file, you must have
specified
> a class in the NXopengroup call - did you leave it blank? It does
have to
> match what's in the file.
>
> Since ISIS are using Matlab quite heavily, I presume Freddie is
writing a
> specific Matlab binding to the API, which should make it easier for
you.
>
> Ray
>
>
>
>> I don't think there is any particular reason why H5 support is not
>> "generic" enough to read files created outside NAPI - it was probably
>> just not allowed for in the original programming, but it should not
be
>> too hard to locate and fix all the "NX_class" attribute dependencies
in
>> the code.
>>
>> Regards,
>>
>> Freddie
>>
>> -----Original Message-----
>> From: tieman [mailto:tieman at aps.anl.gov]
>> Sent: 21 February 2007 15:22
>> To: Akeroyd, FA (Freddie)
>> Cc: nexus at nexusformat.org
>> Subject: Re: [Nexus] Reading generic h5 files.
>>
>> Freddie,
>>
>> Sigh, yes, the proposed code change gets me past the first stumbling
>> block...now the problem is in NX5opengroup where there are a number
of
>> dependencies on the NX_class group attribute. I can probably hack my
>> way around these as well...why is this different for H5 though? HDF4
>> also had group attributes, but I could set the attribute to what I
want
>> and, more importantly, when I read a file there didn't seem to be a
>> required attribute ("NX_class" or otherwise) to properly read the
file.
>>
>> I often read HDF4 files written from sources other than NAPI and, as
>> long as they are SDS and not raster, I can read them just fine.
>>
>> Is there a reason why H5 support is not "generic" enough to read
files
>> created outside NAPI? Does this extend to HDF4 support now? Was I
just
>>
>> getting lucky with reading genereic HDF4 files in the past?
>>
>> As far as Matlab goes, if I could read generic H5 files, I would have
no
>>
>> need for bindings...Matlab supports H5 native. Of course the files
>> would not be "Nexus" compliant without the bindings or mimic thereof
>> within Matlab. But even if I wanted Nexus compliance, I'm probably
>> better off binding to my "Nexus wrapper" libraries that are somewhat
>> higher level than NAPI and much more familiar to me :) In either
case,
>> I have to deal with HDF4 and HDF5 files created outside of Nexus
often
>> enough that this is a limiting issue...
>>
>> Brian
>>
>> Akeroyd, FA (Freddie) wrote:
>>
>>> Brian,
>>>
>>> "NX_class" is an attribute and is attached to each group to indicate
>>>
>> its
>>
>>> NeXus class (NXentry, NXinstrument etc.). If you remove the lines
that
>>> try to read this attribute and instead just have something like
>>>
>>> strcpy(nxclass, "NXunknown")
>>>
>>> Does that help?
>>>
>>> matlab bindings for NeXus would be another way around this problem
...
>>>
>>> I have that on my list of things to do in the next month.
>>>
>>> Regards,
>>>
>>> Freddie
>>>
>>> -----Original Message-----
>>> From: nexus-bounces at nexusformat.org
>>> [mailto:nexus-bounces at nexusformat.org] On Behalf Of tieman
>>> Sent: 20 February 2007 21:18
>>> To: nexus at nexusformat.org
>>> Subject: [Nexus] Reading generic h5 files.
>>>
>>> Hi all,
>>>
>>> Can Nexus (3.0.0) read "generic" h5 files that were not written with
>>>
>> the
>>
>>> Nexus API. Experimentation tells me the answer is "no" but I am
>>>
>> holding
>>
>>> out hope that I am doing something wrong...
>>>
>>> I need to exchange data between a process I've written that uses
Nexus
>>>
>>> to read/write HDF5 files and Matlab. Matlab can read the files I
>>> generate just fine, however, I can not read the files Matlab
>>>
>> generates.
>>
>>> We took an h5 file I wrote with nexus and read it into Matlab
without
>>>
>> a
>>
>>> problem. We then rewrote the file from within Matlab, but my Nexus
>>> program can not read it.
>>>
>>> I think I've managed to trace part of the problem to NX5GetNextEntry
>>>
>> and
>>
>>> the following lines:
>>>
>>> attr1 = H5Aopen_name(grp, "NX_class");
>>> type=H5T_C_S1;
>>> atype=H5Tcopy(type);
>>> H5Tset_size(atype,128);
>>> iRet = H5Aread(attr1, atype, data);
>>> strcpy(nxclass,data);
>>>
>>> I don't understand h5 very well yet but it looks to me like this
>>>
>> section
>>
>>> of code is looking for a "group type" of "NX_class" which Matlab is
>>> unlikely to have written and so the later call to H5Aread fails. I
>>>
>> see
>>
>>> NX5MakeGroup seems to attach the "NX_class" type when the file is
>>> created.
>>>
>>> Is there a way around this? I have a strong need to inter-operate
>>>
>> with
>>
>>> Matlab and potentially additional sources of h5 data but I already
>>>
>> have
>>
>>> a large codebase built around NAPI that I'd love to leverage as
well.
>>>
>>> Any thoughts?
>>>
>>> Thanks!
>>>
>>> Brian Tieman
>>> Advanced Photon Source
>>> Argonne National Laboratory
>>>
>>>
>>> _______________________________________________
>>> NeXus mailing list
>>> NeXus at nexusformat.org
>>> http://lists.nexusformat.org/mailman/listinfo/nexus
>>>
>>> _______________________________________________
>>> NeXus mailing list
>>> NeXus at nexusformat.org
>>> http://lists.nexusformat.org/mailman/listinfo/nexus
>>>
>>>
>> _______________________________________________
>> NeXus mailing list
>> NeXus at nexusformat.org
>> http://lists.nexusformat.org/mailman/listinfo/nexus
>>
>>
>
>
_______________________________________________
NeXus mailing list
NeXus at nexusformat.org
http://lists.nexusformat.org/mailman/listinfo/nexus
More information about the NeXus
mailing list