[Nexus] question about NeXus classes

Wed Aug 27 15:29:12 BST 2008

Dear Darren,

Darren Dale wrote:
> I think my last message might have been overlooked since I posted it on a 
> weekend. I'd like to try to revive the discussion and ask a couple more 
> questions:
>
> On Saturday 26 July 2008 09:42:04 am Darren Dale wrote:
>   
>> On Friday 25 July 2008 13:35:13 Peterson, Peter F. wrote:
>>     
>>> -----Original Message-----
>>> From: nexus-bounces at nexusformat.org
>>> [mailto:nexus-bounces at nexusformat.org] On Behalf Of Darren Dale
>>>
>>> I have a question regarding specialization of the generic class
>>> definitions listed at  http://www.nexusformat.org/Design#NeXus_Classes .
>>> Is it acceptable to create a subclass of NXdetector that is specific to a
>>> single- or multi-element energy-dispersive detector, called NX_Vortex90EX
>>> for example, and save that class name to NX_class? If that is not
>>> advisable, I guess I could use an additional attribute like NX_subclass.
>>> I'm just trying to get asense of the intended use of the NX_class
>>> attribute and its limitations.
>>>       
>>> Thank you for the question.
>>>
>>> The current philosophy of classes is that they are to be thought of as
>>> dictionaries. If you are familiar with xml schema (xsd) the "any" tag is
>>> appropriate. This means that there is one NXdetector which covers all
>>> the various types of detectors in existence. If a particular detector
>>> needs another field that isn't in the current NXdetector, then it should
>>> be added. Subclassing is not an idea that can be applied to the base
>>> classes.
>>>       
>> Thanks for responding. I have almost no experience with xml, so please
>> excuse my ignorance: if I need additional fields, I can add them, but I
>> should continue to use the existing classes. Is that correct?
>>
>>     
>>> However, the NIAC has noted the similarities in several of classes and
>>> applying the object oriented design techniques such as subclassing. The
>>> committee agreed that this is the direction for the future, but are
>>> waiting until the current definitions (v2.0) can be finalized.
>>>       
>
> Is there any news on this subject?
>
>   
Nope, the NIAC will meet end of october. And decisions will only be made 
then. Also, there
is a problem: backwards compatability. An OO scheme makes many things 
easier but files would
look different (other classes) and we cannot make our minds up to do 
this. Also, there is the small
matter of the right object model. The last time we agreed that we do a 
python-API and some people
play around with that one in order to try various object models.
>> I'd like to explain what I'm working on. I have been developing a program
>> for acquisition and analysis at my beamline, and have been generating hdf5
>> files using pytables. I am now working on updating my data model to format
>> the files according to the nexus guidelines, and am starting at the
>> foundation: an object-oriented interface to the hdf5 file and tree. This
>> foundation is not built on the nexus api or the official nexus python
>> bindings, nonetheless I want to make sure my files adhere to the standard.
>> For example:
>>
>> t = NXfile('NXtest.h5')
>> vortex = t['entry']['instrument']['vortex']
>>
>> That currently inspects the node's attributes and returns an instance of
>> the NXdetector python class. In the future I can also inspect the nodes
>> "type" and "description" attributes to find a more specialized constructor
>> if necessary, one that provides additional methods and a richer interface
>> to the same underlying NXdetector data.
>>     
>
> The basic foundation of my project is essentially in place, thanks to the new 
> h5py project: http://h5py.alfven.org/ , which provides a simple and intuitive 
> interface to the hdf5 library. h5py similar to (and somewhat deriving from) 
> pytables, but unlike pytables h5py supports links and is threadsafe. I think 
> it is a great foundation for building an interface geared towards working 
> with NeXus data interactively in python, or with simple python scripts.
>
> I'm not trying to interfere with the official nexus python bindings, which 
> provide a NAPI-ish interface. But I do see an opportunity that I felt I had 
> to explore. I hope nobody feels like I am stepping on anyone's toes. 
>
>   
You only step on our toes if you distribute your stuff as a standard
python-API.Also, we do not care how you write your files. As long as
they adhere to NeXus principles and have the right structure and
the right attributes, we do not care.  We partially decided to keep
the python-API napish, as you call itt, in order to keep similar to
previous work: the Java-API. I am actually the guilty party for the
Java-API and I thought long and hard about a more OO-API at the
time, with objects representing groups and SDS etc.  I decided against
it because the HDF libraries are state machines of their own right
and keeping the OO-machines in sync with HDF libraries seemed like
a nightmare to me.

>> I'm just getting started, so right now I can access the data in a pythonic
>> way and return basic NX* object instances, but the objects are all
>> featureless aside from providing access to the underlying data. Since I'm
>> just getting started, it would be helpful if you could share any
>> information concerning the future of nexus' definitions and organization.
>> Are the committee's discussions available on the web? Do you think my
>> approach is one that the nexus developers could be comfortable with, or
>> should I be considering another approach? My hope is that the project would
>> eventually be useful to the wider community.
>>     
>
>   
Look in the NeXus wiki at: www.nexusformat.org in
Discussions/Design Issues/Object Oriented Nexus. Feel free to comment.

> Today I discovered nxvalidate in the nexus trunk and tried running it on some 
> of the test files in test/data. They either did not conform to the new 3.0 
> schema or the perhaps the schema does not include all the definitions needed 
> to validate these files. nexus_defs/schema/testfile.xml does validate. Is 
> there any additional information available concerning the new organization of 
> NeXus classes/subclasses, new features, etc?
>   
The files in test/data are there to test some mechaniques of the API. They
do not conform to the standard. I apologize for this, but NeXus is 
maintained by very
people in their spare time and thus we are not always as consistent as 
would be
desirable. But worry though( and try to resolve the issue) if your files 
fail the nxvalidate
test.
> At my beamline, we do some scanning X-ray fluorescence microscopy, and we 
> process the data in real time. I would like to group the processed data 
> together as a child group of the NXentry, something like:
>
> NXentry/element_maps/NXdata for each element modeled
>
> It looks like the class I am looking for to provide the element_maps grouping 
> is described in NXprocessed.xml or PROCESSED.xsd, which is not present in the 
> class list at http://www.nexusformat.org/Design. I am still trying to get my 
> bearings, can NXentry contain a NXprocessed group and still conform, or does 
> NXprocessed have to exist under NXroot? Can NXprocessed contain additional 
> metadata besides what is listed in NXprocessed.xml, like links to NXdetector 
> MCA data, or should that information be stored in NXprocessed/NXprocess? Is 
> there documentation answering basic questions like these?
>
>   
We are working on the doc bit. And to answer the question: the general idea
was to store processed data in a separate entry. With NXprocess holding any
info on how the processing was done. Such an entry would have links to 
the admin
information in the raw data entry and new data fields for the processed 
data. And an
NXprocess to describe what has been done.  Yes, an entry with NXprocess 
is conformant,
IMHO.

          Best Regards,

                      Mark Koennecke

> thanks!
>
> Darren
> _______________________________________________
> NeXus mailing list
> NeXus at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus
>   