[Nexus-developers] Python Tree API

[Ray Osborn]

Dear colleagues,
I have been working extensively on the Python tree API following the recent NeXus code camp at Argonne that involve some major structural changes that I think make the code more robust and convenient, so I think it is time to get some feedback from other NeXus developers. There are two issues - one is whether these changes are the right way to go and the other is how backwardly compatible this needs to be. The Python port of NAPI has not been changed, so I am only talking about tree.py, which I suspect has not been used that much since it is still not officially a part of the NeXus distribution. Python is flexible enough that it may be possible to map much of the old API onto the new one, but I only want to do that if it is really necessary. However, because the changes are so substantial, I have not loaded this on to the Subversion server. If I get enough encouragement to do this, I am happy to do so. In the meantime, I attach a tar file with the nexus subdirectory. I think that if NEXUSLIB is defined then 'import nexus' should work if you put the package in your site-packages directory.

I don't know the schedule for the next NeXus release, but I suspect that we won't be able to get this in, but I would really appreciate it if you can find the time to check this out, and send some feedback. We may need to check for some backward compatibility issues, although Paul and I thought it unlikely that many people have been using the tree part of the API. The napi part I haven't touched.

I have tried to document the changes in the docstrings, so the simplest thing is probably to read what I have written particularly for the NXfield (formerly SDS) and NXgroup classes. I would be interested how the documentation looks in doxygen.

To summarize, the main change is to explicitly place the NeXus items (NXfields or NXgroups) within each group into a dictionary named 'entries', but to use the __getattr__ and __setattr__ methods to preserve the convenience of the direct syntax. 

i.e., In the example below, after assigning the NXgroup, the following four NeXus object assignments are all equivalent:

>>> entry.sample = NXsample()
>>> entry.sample.entries['temperature'] = NXfield(40.0,name='temperature')
>>> entry.sample.temperature = NXfield(40.0)
>>> entry.sample.temperature = 40.0

There are a few internal Python attributes that have special meaning, e.g., 'name', 'group', 'path', so if a NeXus NXfield needs to be called 'group', then it has to be entered into the 'entries' dictionary explicitly (as in the first example above) or using the insert method:

>>> entry.sample.insert(NXfield(40.0,name='temperature'))

This saves typing the name twice. The number of such attributes is pretty small - at the moment, they are 'name', 'group', 'entries', 'attrs', 'dtype','shape', 'link', 'path', and 'head'. I don't think I have seen any of those defined within a NeXus NDL file (I'm not sure about 'shape'), but it gives us a way of coping with them if there is a name clash. 

This means that we can relax the requirement that all the methods start with 'nx'. Now, we have the 'tree' method, instead of 'nxtree'. Only 'nxdata' and 'nxaxes' are kept, because they are both common names used in NeXus files.

In particular, the NXfield attributes can have the same name as the standard Numpy attributes (e.g., 'reshape', not 'nxreshape'). 

Furthermore, we can make is so that all the Numpy ndarray attributes work as well, using the __getattr__ again. So now, we can use them like this. 

>>> x=NXfield(np.linspace(0,100.,11),name='x')
>>> x.size
       4
>>> x.sum()
       10.0
>>> x.max()
       4.0
>>> x.mean()

Paul also showed me how to cast NXfields as ndarrays, using the __array__ method, so we can also do other operations.

>>> np.sin(x)
       array([ 0.84147098,  0.90929743,  0.14112001, -0.7568025 ])

I think this gets as close as possible to making NXfields true subclasses of ndarrays, without major surgery. 

I have done the same trick with NeXus attributes, which are stored in the 'attrs' dictionary, with class NXattr. This make NXgroup attributes work the same way as NXfields. In both cases, I think we could recommend that anyone writing scripts should use the wordier dictionary assignments to be completely safe. The shorter versions are to make it more convenient and intuitive for people in an interactive session, and are important, in my opinion, to people finding this attractive to use. 

The docstrings are quite extensive, although they are not complete. Let me know if there are things that really need further explanation. 

Thanks,
Ray
-- 
Ray Osborn
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nexus.tar.gz
Type: application/x-gzip
Size: 39523 bytes
Desc: not available
URL: <http://lists.nexusformat.org/pipermail/nexus-developers/attachments/20111109/fbdefcc5/attachment-0001.gz>
-------------- next part --------------