[Nexus-developers] NeXus-API XML in alpha

[Mark Koennecke]

  High,

  the last argument against NeXus: "It is not ASCII, I cannot edit my data"
  is crumbling.

  The NeXus-API for XML passes the standard NeXus-API self test which at
  least promotes it to alpha. Contrary to my previous e-mail, I decided
  to base the XML part of the API on mini-XML, a tree API for XML. The
  mxml homepage:
            http://www.easysw.com/~mike/mxml
  I worked with the author of mxml, Michael Sweet, to extend mxml in such
  a way that we can store NeXus datasets in the tree efficiently. I
  very much hope, that my changes will make it in the standard
  distribution of mxml; so that we do not have to maintain that
  library as well.  

  For links I took the liberty to store them as elements of type NXlink
  with an attribute target which is the path to the linked item. NXopengroup,
  NXopendata, etc follow these links. The implementation of the
  NeXus-API is next to complete, but two issues remain:
  - I did not implement unlimited dimensions. This would be complicated,
    and I am not convinced that we need it.
  - Currently, the XML-API rejects non text array type attributes and
    converts single number attributes silently to text. Reading attributes
    returns only text attributes. If this is not acceptable, we need to
    store type information with the attribute. I would suggest a scheme 
    like some_attribute="NX_INT32[10]: 1 2 3 4 5 6 7 8 9 10". What does
    everybody else think?
  The XML NeXus API can be enabled through the define: NXXML.

  I also fixed the issue with NXIReportError which ocurred when the 
  different files of the API were compiled separatly. This was a
  prototyp problem which I solved by putting some more things into
  napi.h. All the NeXus-API bits can now be compiled separatly.

  Peter, long ago, reported an issue to me about performance problems
  when reading multi dimensional arrays from the Java-API. I looked
  into this problem and  I'am afraid this can only be fixed by
  introducing another performance issue, a memory problem now:
  Currently, the Java-API already needs two copies of the data for
  conversions from NeXus to Java number formats. If we fix the
  performance problem we need three:
  - the byte arrya coming from the C-API
  - An intermediate one dimensional array
  - The final multi dimensional array.
  I am not yet ready to do this.  I think the implementation in 
  HDFArray.java which converts rows of multi dimensional data sets is 
  the best we can do in the general case. Has anybody got a better idea?
  
  The autoconf stuff does not work again. Even on the upcoming PSI
  Linux distribution, Scientific Linux 3.2 (which is Redhat Enterprise
  3.2) it does not work. I think the problem is that the autoconf
  stuff was developed on a bleeding edge Linux distribution. For
  portability, may be, one should develop autoconf scripts on the oldest
  machine available. I still have a Redhat 6.2 machine. Perhaps we can
  resolve this issue during the NIAC meeting when all the experts are
  here. PSI also provides a sizable number of test platforms. I am
  quite sure that the problem is with the scripts. Normally configure
  scripts run on my machines. They usually complain because a library
  is too old, but this is a completely unrelated matter.

  Back to NeXus-XML: If nobody objects, I will tag the current
  version in the CVS as 2.1 and upload the new stuff which will then
  become NeXus-3.0 after some more testing by you. 
  
                   See you soon,

		       Mark Koennecke