[NeXus-committee] NeXus XML format and reduced data files

[Mark Koennecke]

Akeroyd, FA (Freddie) schrieb:
> The Small Angle Scattering (SAS) community have been working on a
> reduced data file format and I enclose an example (tk49.xml) of what
> they seem to be converging on. You'll notice that the file looks very
> simililar to a NeXus XML file, but there is an important difference on
> how the numerical Q, intensity, and error data is stored. 
>
> The current NeXus XML interface stores all the values of an array
> together under one XML tag - this makes it harder to read the data into
> other programs that extract logical columns of numbers. The enclosed
> tk49.xml satisfied this by having the numbers in fixed format column
> locations, but also surrounded by XML tags that allow easy extraction
> into named Excel columns (see http://www.ill.fr/lss/canSAS/XML_SAS.htm).
> Note that this webpage shows Excel 2003 - the latest Excel 2007
> automatically recognises the file structure and puts up a Task Pane from
> which the user can browse for the data or metadata they want.
>
> data_comparisons.xml compares the current NeXus XML format with a
> proposal for an alternative XML representation based on the ideas above.
> I have had a look at the NeXus XML source code and, with some work, it
> should be possible to support both this and the current XML format. I
> believe it would be advantageous to NeXus if the SAS community were to
> use the NeXus API with their reduced data files - they will also gain by
> getting access to NeXus (and HDF) tools to manipulate their reduced
> data.
>
> What do people think about the API supporting a second XML format? The
> SAS community will be discussing data formats at the upcoming canSAS-V
> meeting (http://www.smallangles.net/canSAS/schedule.html) - if we wish
> to collaborate on a second XML representation that meets their specific
> needs while maintaining generality we will need to decide soon.
>
>   
I think that this is rather like a lot of work.  Especially  if you wish 
to port similar structures to HDF-5 and HDF-4.
I also think that there is another question folded into this:
  - Do we want a table data type? Because that is, IMHO, what this 
proposal is.

Though I myself do not fancy excel for data analysis I know that many 
people do. I wonder about a couple of things:
- Can this SAS be converted to NeXus and back using a XLST transform?
- May be we rather suggest a NXU function which takes as input a path to 
the group and, given a suitable structure
  of the group, emits its content as a CSV file. CSV is understood by 
many database and spreadsheet products,
  including excel.

                       Have a nice weekend,

                                   Mark