Design issues for extension of NeXus for treated data

[Tanya Maria Riseman]

 Hi all,

	Good points, Mark.

	If the NeXus API people are willing to add the ability to
 store "treated" data and the results from data reduction,
 there will be a pressure to 
 1. Increase the number of levels of groups, that is make the 
    data format less flat than it is currently.
    That will make it harder to understand and scare off new users.
and/or
 2. Somehow have a scheme to keep track of which files the data reduction
    comes from, either informally using file names (not maintained)
    or using file "mountings" in HDF 5. As Mark Koennecke points out, 
    maintaining file mountings will be a nightmare.


	Unless the storage of "treated" data and the results from data reduction
 is done in a SUPERIOR way, there is almost no point in doing at all, because 
 everyone has their own favorite "good-enough" programs like Origin. 
 People will prefer to store in the format which matches their program.
 By superior I mean seemlying-incompatible features like asymmetric error 
 bars, covariance matrices, ease of use and a nice ASCII version that 
 allows most simple cases to be read directly into spreadsheets. 
 (XML and HTML are tedious to edit.) Although it is a nice idea in principle
 to have one format usable by many programs.

	I think that it is better to keep the NeXus defintion completely 
 separate from the reduced and treated data defintion ("Ratdata"?).
 Maybe 3 sections: NeXus, Ratdata and code/tools common to both.
 We need a nicer name than Ratdata.
 That way the UNION of the two could be used for storing analysis with 
 raw data for those people who like that (I personally think it is a bad idea)
 or for storing analysis with selected info copied from the NXSample
 and NXInstrument (OK). And people who don't use NeXus, in other fields, 
 might like to use just the "Ratdata" part. 

	Is there the collective will to design and impliment Ratdata?
 Has something similar already been written for HDF or similar package?

	- Tanya

-- 
  Dr. Tanya Riseman
  School of Physics and Astronomy	phone 44-121-414-7322
  University of Birmingham		fax   44-121-414-4719
  Birmingham B15 2TT, United Kingdom	email: tmr at th.ph.bham.ac.uk