[Nexus] Status of NAPI

Mon Jun 22 05:53:08 BST 2015

I’m not in a position to state what the official policy is - I don’t think it was officially discussed as part of the last NIAC agenda, but I was sure I heard the idea of a ‘NAPI Light’ mentioned. My impression at the time was that this had been agreed earlier, perhaps in one of the telco meetings, but I may have misunderstood what was said. It wouldn’t be the first time.

I don’t want to prescribe what people use to write NeXus files - if you are comfortable using the HDF5 libraries and are careful to follow the NeXus guidelines, then that’s great. However, I do think that the simplification of the NAPI was a great help to people like me, i.e., people who do some programming in their spare time. I am a little concerned that the governance of NeXus now only caters to those whose main responsibility is software development rather than to less “professional” scientist/programmers. That was the reason I protested so vociferously about the unnecessary complications introduced at the last NIAC in the NXdata axis definitions. If you program all the time, this extra complexity is no big deal, but if you are trying to write a program to do a quick and dirty piece of data analysis, as scientists often do, then it becomes a problem. I think NeXus should cater to both.

On the other hand, scientists probably prefer to use Python anyway, so perhaps the NAPI doesn’t matter any more. As to whether to use h5py or the nexusformat package (which is written using h5py), that’s just a matter of taste.  Personally, I prefer to write something like:

>>> NXdata(z, (y,x)).save('data.h5')

rather than:

>>> f = h5py.File('data.h5','r+')
>>> f.create_group('entry')
>>> f['entry'].attrs['NX_class'] = 'NXentry'
>>> f['entry'].create_group('data')
>>> f['entry/data'].attrs['NX_class'] = 'NXdata'
>>> f['entry/data'].attrs['signal'] = 'z'
>>> f['entry/data'].attrs['axes'] = ['y', 'x']
>>> f['entry/data'].attrs['y_indices'] = 0
>>> f['entry/data'].attrs['x_indices'] = 1
>>> f['entry/data'].create_dataset('x', data=x)
>>> f['entry/data'].create_dataset('y', data=y)
>>> f['entry/data'].create_dataset('z', data=z)
>>> f.close()

but both produce the same result.

Ray

On Jun 21, 2015, at 3:37 PM, Andy Gotz <andy.gotz at esrf.fr<mailto:andy.gotz at esrf.fr>> wrote:

Ray,

does this mean the the goal is still to propose NAPI as the official way to write Nexus files? Developing and maintaining an API is a lot of work. I wonder if a community like Nexus which has very limited resources should take on more work. I remember issues in the past with multi-threading and bugs which make the NAPI difficult to use. Without adequate resources these problems will come back.

Our preference is to use the HDF5 libraries directly even for Python (h5py).

Andy

On 21/06/2015 22:23, Osborn, Raymond wrote:
I had understood that there were plans to create a light version of the NAPI, which was only dependent on HDF5. I am slightly concerned about encouraging people only to use HDF5 directly, because that makes it entirely dependent on them to ensure compliance with the NeXus rules - e.g., by adding a group attribute, NX_class, to each group.

Ironically, NAPI was written to remove much of the complexity in writing files by having a reduced set of functions. HDF5 is better written than HDF4 in this respect, but the additional flexibility still makes it quite daunting to learn. The NeXus API is a model of simplicity so a lightweight API would still make sense.

Ray
P.S. Of course, I would encourage people to use the Python interface (http://nexpy.github.io/nexpy/), which is even simpler, and ensures compliant NXdata groups as well.

On Jun 21, 2015, at 1:04 PM, Tobias Richter <Tobias.Richter at esss.se<mailto:Tobias.Richter at esss.se>> wrote:

Hi Andy,

The official policy just reflects the (perceived) requirements of the community. Your thoughts are correct NAPI is not recommended for new applications. And yes, the manual hasn’t been updated in all areas to reflect this. Please point us to the relevant sections you find unclear. Someone reading the texts with fresh eyes can really make a difference.

Since we focus on HDF5 now, the NAPI backend abstraction (supporting XML and HDF4 in addition) isn’t needed for that. NAPI also provides nothing that would ensure files follow the standard and very little that would at least make conforming easier. So the only benefit is some slight simplification of the hdf5 interface, which denies some of the power of the underlying container format.

That being said, NAPI is not disappearing any time soon, while people rely on it. We should drop support for unused bindings though.

In my view in the long run, some reference application or tools would still be needed to be able to demonstrate interoperability.

Regards,

Tobias

On 21 Jun 2015, at 17:39, Andy Gotz <andy.gotz at esrf.fr<mailto:andy.gotz at esrf.fr>> wrote:

Hi Eugen + Nexus community.

I have a naive question : what is the status of the NAPI? I thought it is being maintained only for legacy applications and that it is not recommended for new applications. But this is not clear from the manual. In some places the reader gets the impression the NAPI is the official API for accessing files conforming to Nexus. In other places the opposite is stated and the hdf5 API is considered as the right way to access files.

What is the official policy on the NAPI?

Andy
_______________________________________________
NeXus mailing list
NeXus at nexusformat.org<mailto:NeXus at nexusformat.org>
http://lists.nexusformat.org/mailman/listinfo/nexus

_______________________________________________
NeXus mailing list
NeXus at nexusformat.org<mailto:NeXus at nexusformat.org>
http://lists.nexusformat.org/mailman/listinfo/nexus

--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>

_______________________________________________
NeXus mailing list
NeXus at nexusformat.org<mailto:NeXus at nexusformat.org>
http://lists.nexusformat.org/mailman/listinfo/nexus

_______________________________________________
NeXus mailing list
NeXus at nexusformat.org<mailto:NeXus at nexusformat.org>
http://lists.nexusformat.org/mailman/listinfo/nexus

--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nexusformat.org/pipermail/nexus/attachments/20150622/5079a45f/attachment-0001.html>