[Nexus] Reading generic h5 files.
tieman
tieman at aps.anl.gov
Wed Feb 21 19:58:22 GMT 2007
Freddie, Ray,
I figured the issue was something like this. For HDF4, I do create a
"type"--if it's a group that is defined in Nexus, I try and use the NX*
type (we don't have many of these), for the rest, I make something up
that would make sense to someone who might read it. When reading the
data back in, for the most part, I just pass the type along (the actual
value is quarried from the file) with the name and everyone seams to be
happy.
For HDF5, the missing NX_class attribute signals errors which ultimately
cause my code to fail. I agree that modifying the NAPI to be more
forgiving may cause problems elsewhere and maybe we can fix this in the
Matlab code. Is it really as simple as adding an "NX_class" attribute
to all groups? I did an h5dump on a file written by NAPI and I do not
see a "NX_class" attribute in the top level group "/" but I do see it in
all the other groups. Do I need to have a "Nexus_version" attribute in
"/"? Do any of the fields have special attributes that are needed?
Matlab bindings may or may not be of use to us. As you probably well
know, we only pretend to write Nexus files at the APS ;) I've thought
many times about dropping the pretense and modifying my NAPI wrapper
code to call HDF directly to avoid this confusion, but I've got a lot of
code that relies on NAPI to read/write HDF files. Nexus as an API has
served me very well over the years! If we can get Matlab to fake what
NAPI needs, we can continue to get by for now...at least until someone
with more clout than I snaps their fingers and declares the APS WILL use
Nexus as a file format...
Thanks!
Brian
Ray Osborn wrote:
> On 2007/02/21 11:53, "Akeroyd, FA (Freddie)" <F.A.Akeroyd at rl.ac.uk> wrote:
>
>
>> Brian,
>>
>> In HDF4 objects had the concept of a "class" and this could be used by
>> the NeXus API to store the data type; a separate group attributes was
>> not needed. In HDF5 I believe there was no such built in facility and so
>> an "NX_class" group attribute had to be created and used instead.
>>
>
> Brian,
> The original idea with the NeXus API was that it would only be used to read
> and write NeXus-standard-conforming files, which required that each group
> had a name and a class associated with it. The class needed to have the
> prefix "NX", so NAPI was only set up to read such groups. The rest would be
> ignored - they could be there, but the NAPI did not know how to handle them.
> HDF4 had a class automatically associated with each group, which was
> simulated in HDF5 by adding an NX_class attribute.
>
> I guess the API could be rewritten to be a bit more forgiving when reading a
> non-standard file, but it might cause problems. Mark would be best
> qualified to comment on that. Probably the simplest thing is to output the
> Matlab data with a script that adds the NX_class group attribute, making
> sure it started with "NX" to be safe.
>
> I presume that the old HDF4 files worked because HDF4 automatically gave
> each group a class. When NAPI looked for it, it found it - the HDF4 version
> doesn't seem to care whether the class has the right prefix or not.
> However, the HDF5 NAPI looks for the NX_class attribute, and complains if
> it's not there. When you read the HDF4 Matlab file, you must have specified
> a class in the NXopengroup call - did you leave it blank? It does have to
> match what's in the file.
>
> Since ISIS are using Matlab quite heavily, I presume Freddie is writing a
> specific Matlab binding to the API, which should make it easier for you.
>
> Ray
>
>
>
>> I don't think there is any particular reason why H5 support is not
>> "generic" enough to read files created outside NAPI - it was probably
>> just not allowed for in the original programming, but it should not be
>> too hard to locate and fix all the "NX_class" attribute dependencies in
>> the code.
>>
>> Regards,
>>
>> Freddie
>>
>> -----Original Message-----
>> From: tieman [mailto:tieman at aps.anl.gov]
>> Sent: 21 February 2007 15:22
>> To: Akeroyd, FA (Freddie)
>> Cc: nexus at nexusformat.org
>> Subject: Re: [Nexus] Reading generic h5 files.
>>
>> Freddie,
>>
>> Sigh, yes, the proposed code change gets me past the first stumbling
>> block...now the problem is in NX5opengroup where there are a number of
>> dependencies on the NX_class group attribute. I can probably hack my
>> way around these as well...why is this different for H5 though? HDF4
>> also had group attributes, but I could set the attribute to what I want
>> and, more importantly, when I read a file there didn't seem to be a
>> required attribute ("NX_class" or otherwise) to properly read the file.
>>
>> I often read HDF4 files written from sources other than NAPI and, as
>> long as they are SDS and not raster, I can read them just fine.
>>
>> Is there a reason why H5 support is not "generic" enough to read files
>> created outside NAPI? Does this extend to HDF4 support now? Was I just
>>
>> getting lucky with reading genereic HDF4 files in the past?
>>
>> As far as Matlab goes, if I could read generic H5 files, I would have no
>>
>> need for bindings...Matlab supports H5 native. Of course the files
>> would not be "Nexus" compliant without the bindings or mimic thereof
>> within Matlab. But even if I wanted Nexus compliance, I'm probably
>> better off binding to my "Nexus wrapper" libraries that are somewhat
>> higher level than NAPI and much more familiar to me :) In either case,
>> I have to deal with HDF4 and HDF5 files created outside of Nexus often
>> enough that this is a limiting issue...
>>
>> Brian
>>
>> Akeroyd, FA (Freddie) wrote:
>>
>>> Brian,
>>>
>>> "NX_class" is an attribute and is attached to each group to indicate
>>>
>> its
>>
>>> NeXus class (NXentry, NXinstrument etc.). If you remove the lines that
>>> try to read this attribute and instead just have something like
>>>
>>> strcpy(nxclass, "NXunknown")
>>>
>>> Does that help?
>>>
>>> matlab bindings for NeXus would be another way around this problem ...
>>>
>>> I have that on my list of things to do in the next month.
>>>
>>> Regards,
>>>
>>> Freddie
>>>
>>> -----Original Message-----
>>> From: nexus-bounces at nexusformat.org
>>> [mailto:nexus-bounces at nexusformat.org] On Behalf Of tieman
>>> Sent: 20 February 2007 21:18
>>> To: nexus at nexusformat.org
>>> Subject: [Nexus] Reading generic h5 files.
>>>
>>> Hi all,
>>>
>>> Can Nexus (3.0.0) read "generic" h5 files that were not written with
>>>
>> the
>>
>>> Nexus API. Experimentation tells me the answer is "no" but I am
>>>
>> holding
>>
>>> out hope that I am doing something wrong...
>>>
>>> I need to exchange data between a process I've written that uses Nexus
>>>
>>> to read/write HDF5 files and Matlab. Matlab can read the files I
>>> generate just fine, however, I can not read the files Matlab
>>>
>> generates.
>>
>>> We took an h5 file I wrote with nexus and read it into Matlab without
>>>
>> a
>>
>>> problem. We then rewrote the file from within Matlab, but my Nexus
>>> program can not read it.
>>>
>>> I think I've managed to trace part of the problem to NX5GetNextEntry
>>>
>> and
>>
>>> the following lines:
>>>
>>> attr1 = H5Aopen_name(grp, "NX_class");
>>> type=H5T_C_S1;
>>> atype=H5Tcopy(type);
>>> H5Tset_size(atype,128);
>>> iRet = H5Aread(attr1, atype, data);
>>> strcpy(nxclass,data);
>>>
>>> I don't understand h5 very well yet but it looks to me like this
>>>
>> section
>>
>>> of code is looking for a "group type" of "NX_class" which Matlab is
>>> unlikely to have written and so the later call to H5Aread fails. I
>>>
>> see
>>
>>> NX5MakeGroup seems to attach the "NX_class" type when the file is
>>> created.
>>>
>>> Is there a way around this? I have a strong need to inter-operate
>>>
>> with
>>
>>> Matlab and potentially additional sources of h5 data but I already
>>>
>> have
>>
>>> a large codebase built around NAPI that I'd love to leverage as well.
>>>
>>> Any thoughts?
>>>
>>> Thanks!
>>>
>>> Brian Tieman
>>> Advanced Photon Source
>>> Argonne National Laboratory
>>>
>>>
>>> _______________________________________________
>>> NeXus mailing list
>>> NeXus at nexusformat.org
>>> http://lists.nexusformat.org/mailman/listinfo/nexus
>>>
>>> _______________________________________________
>>> NeXus mailing list
>>> NeXus at nexusformat.org
>>> http://lists.nexusformat.org/mailman/listinfo/nexus
>>>
>>>
>> _______________________________________________
>> NeXus mailing list
>> NeXus at nexusformat.org
>> http://lists.nexusformat.org/mailman/listinfo/nexus
>>
>>
>
>
More information about the NeXus
mailing list