[Nexus] Correct way to specify multiple signals

[V. Armando Sole]

On 11.01.2018 18:15, Osborn, Raymond wrote:
> I have tried to read all the other emails more carefully, and I am
> worried that we are conflating two different things and we need to
> focus on which one we really are trying to solve.
> 
> 1) As I understand it, the wish to provide an auxiliary signal is
> motivated by cases where two separate sets of data are closely linked,
> e.g., the raw data and a fit to that raw data. If we concentrate on
> that one example, then the desire is to provide a way to plot both
> together in some automated way, i.e., a plot of one will frequently
> (usually?) be combined with an overplot of the other. As an
> experimentalist, I have concerns about putting both of these in a
> single NXdata group.
> 
> Although the raw data is immutable, fits are not. I spend half my life
> trying out new fits, and I don’t think there is any case where I
> would want to store the “one true fit” in the same NXdata group as
> the raw data. Next week, I will change my mind. If that is the prime
> motivation, then I think Armando may be right to suggest some
> modification of the NXprocess group, which could contain both the raw
> data and the fit in some linked way. However, the raw data should
> still be stored in the parent NXentry group as the “result" of the
> experiment. The NXprocess group would then contain one possible way of
> analyzing it.

Because of all that and even more, in my ESRF proposal the NXprocess 
group also contains the configuration used by the analysis program with 
the goal of having full provenance. Nobody limits the number of 
NXprocess groups in a NeXus file and one can have as many as desired 
corresponding to different configurations and so on.

Please note that nothing of my proposal as it is brakes "NeXus" rules. 
NXcollection and NXdata groups are allowed everywhere except at root 
level. So there is no "modification" of NXprocess required.

The only thing needed to have the solution fully compliant with NeXus is 
to allow multiple signals.

I find the discussion has been very fruitful because it has allowed to 
get the feedback from Peter Chang. If I entered in the NXprocess 
discussion was to explain Jacob how we deal with problems as the one he 
asked.

If you want to endorse something similar to what currently is just an 
ESRF internal proposal that at the same time is NeXus compliant as an 
official use then it needs discussion. To endorse that use of NXprocess 
is NOT the goal of my post as NXprocess as it is allows me to do what I 
want. What I need is an official way to use multiple signals so that use 
cases as the one presented here and many others are "official". If we 
cannot get it, then that point will be the only point in our usage of 
NXprocess that is not NeXus compliant in our approach.

> 2) The other motivation, according to what Armando has written is to
> provide guidance to the user about what should be plotted against
> what, perhaps within a program like PyMCA or NeXpy.

That can of worms was opened by Ben or by you, not me. I said one cannot 
anticipate all the possible user choices and that is outside NeXus 
scope.


> For
> multidimensional datasets, this opens up a can of worms because we
> then have to specify what axes go along with what signal.

That problem was addressed in 2010 and to me is closed and I do not 
intend to re-open it.

Armando