[Nexus] Correct way to specify multiple signals

[Osborn, Raymond]

I have tried to read all the other emails more carefully, and I am worried that we are conflating two different things and we need to focus on which one we really are trying to solve.

1) As I understand it, the wish to provide an auxiliary signal is motivated by cases where two separate sets of data are closely linked, e.g., the raw data and a fit to that raw data. If we concentrate on that one example, then the desire is to provide a way to plot both together in some automated way, i.e., a plot of one will frequently (usually?) be combined with an overplot of the other. As an experimentalist, I have concerns about putting both of these in a single NXdata group.

Although the raw data is immutable, fits are not. I spend half my life trying out new fits, and I don’t think there is any case where I would want to store the “one true fit” in the same NXdata group as the raw data. Next week, I will change my mind. If that is the prime motivation, then I think Armando may be right to suggest some modification of the NXprocess group, which could contain both the raw data and the fit in some linked way. However, the raw data should still be stored in the parent NXentry group as the “result" of the experiment. The NXprocess group would then contain one possible way of analyzing it. To save space, it is, of course, possible to link the raw data group to the NXprocess group, but that would require the fit to be in a separate NXdata group so that it didn’t contain one fit among many possible fits. The example I gave on Github of fit results generated by lmfit has different NXdata groups representing the data, fit, and each component function included in the fit, along with other parameters. At the moment, I store these in an NXsubentry, but these could easily be added to a redefined NXprocess group, which has a timestamp and therefore represents one possible analysis. I would certainly support storing this kind of information and I would support a debate about the best way to store fits in general.

2) The other motivation, according to what Armando has written is to provide guidance to the user about what should be plotted against what, perhaps within a program like PyMCA or NeXpy. For multidimensional datasets, this opens up a can of worms because we then have to specify what axes go along with what signal. Adding an extra set of signals, either to the ‘signal’ attribute, or to a new ‘signals’ or ‘auxiliary’ attribute doesn’t solve this. If the ESRF has some particular use cases where this is a well-defined problem, then I would support them adding their own private attributes to accommodate them, but we would need to see several examples, perhaps as ‘nexusformat’-style trees, to see if those use cases warrant extending the NeXus standard itself.

By the way, I have not been able to work out what the NXplotxd groups are for. I am almost certain that I would not support it, but I would need to see a clearer description to really judge.

With best regards,
Ray
--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nexusformat.org/pipermail/nexus/attachments/20180111/18e226c5/attachment.html>