[Nexus] Correct way to specify multiple signals

[Osborn, Raymond]

I know I’m being cast as an obstructionist Luddite in this thread, who isn’t able to understand the obvious, but it comes from experience that we make bad decisions when we don’t think through the implications. I am perfectly capable of understanding the desire for multiple signals, but we need to experiment with a number of practical examples before we understand what the consequences of this decision are.

Here’s one that needs clarification. In Pete Jemian’s SPEC importer, he converts meshes into 2D data, producing an NXdata like this:

data:NXdata
 @axes = [‘K’ ‘H']
 @signal = ‘NaI'
 @auxiliary = [‘detector’ ‘ic0’ ‘ic1’ ‘ic2’ ic3’]
 @auxiliary_axes = [[‘K’ ‘H’] [‘K’ ‘H’] [‘K’ ‘H’] [‘K’ ‘H’] [‘K’ ‘H’]]
 Epoch = float64(16x16)
 H = float64(16)
 K = float64(16)
 L = float64(16x16)
 NaI = float64(16x16)
 Ring = float64(16x16)
 Seconds = float64(16x16)
 detector = float64(16x16)
 ic0 = float64(16x16)
 ic1 = float64(16x16)
 ic2 = float64(16x16)
 ic3 = float64(16x16)

As you can see, I added attributes following what I believe is one of the suggestions for allowing extra signals. Since the data is 2-D, the axes are size-2 arrays, so each auxiliary axis also has to be an array of size-2 arrays matching the axes. (By the way, the latest version of the standard requires extra attributes of the form ‘Two_Theta_indices’ stating what dimension they refer to - it’s not an aspect of the standard that I support, but it would make the above example much more complicated).

Note that, without the auxiliary attributes, the above is a NeXus-conforming data set, so it already contains multiple signals and multiple axes.

OK - I think everyone is thinking of 1-D arrays, because it’s difficult to overplot 2-D arrays, but 1D arrays have their difficulties as well.

data:NXdata
 @axes = 'Two_Theta'
 @signal = 'NaI'
 @auxiliary = [‘detector’ ‘ic0’ ‘ic1’ ‘ic2’ ic3’]
 @auxiliary_axes = [’Seconds’, ‘Epoch’, ‘Epoch’, ’Two_Theta']
 Epoch = float64(26)
 H = float64(26)
 K = float64(26)
 L = float64(26)
 NaI = float64(26)
 Ring = float64(26)
 Seconds = float64(26)
 Two_Theta = float64(26)
 detector = float64(26)
 ic0 = float64(26)
 ic1 = float64(26)
 ic2 = float64(26)
 ic3 = float64(26)

Here I have made an arbitrary choice of which x-axis to plot against which y-axis, but what is a generic NeXus program, like NeXpy, meant to do here? Is it meant to provide an option to plot every possible signal against its axis on the same plot, resulting in six different y-axes and four different x-axes, all of which might need to be arbitrarily scaled against each other (or do we provide the scaling as well?). That is not trivial to do in Matplotlib.

My point is that I can see why people want multiple signals, but it may be difficult to define them in a way that would allow be unambiguously parsed in the way that a standard demands. I’m not saying it is impossible, but if we are to extend the standard, these need to be thought through.

Remember that, on your beamlines, there is nothing to stop you adding menu items to your applications that will overplot specific signals that are typically plotted together.

In the last day, I have provided four different explicit examples to show why this might be complicated.  I am extremely willing to be convinced if you provide your own examples that show why this should be a part of the standard.

Ray

On Jan 12, 2018, at 1:14 AM, GOTZ Andrew <andy.gotz at esrf.fr<mailto:andy.gotz at esrf.fr>> wrote:

Armando,

thank you for your patience - your explainations are clear to me! It is a pity the Nexus community do not understand them

Andy


On 12/01/2018 06:54, V. Armando Sole wrote:
On 11.01.2018 23:45, Osborn, Raymond wrote:

But you haven’t explained why you need multiple signals. If it’s
to store the fit with the raw data, then you will also probably need
multiple axes, since fitted functions are normally calculated at many
more points than the raw data. How would you handle that? And do you
propose to store a copy of the fields within the NXentry raw NXdata
group?

I have proposed both and explicitly mentioned that auxiliary_signals and auxiliary_axes would allow to achieve co-plotting raw_data and fitted data with different number of points in my reply to Peter's proposal.

You are focused on the fit, but any data treatment generating more than one "signal" makes desirable to be able to group them. PyMca and NexPy have no problem on visualizing anything, but plenty of developers complain about the fat they are "forced" to create a separated NXdata for each "signal" if they want the end user to inspect the data in a friendly way. The fit case is just so evident that should be enough to justify having that feature.

Other simple use case that I aim to have solved at the ESRF is to achieve the goal of providing the user with a simple way to browse the entries of an HDF5 file generated during the acquisition and presenting them what they had at acquisition time. As you know sometimes (even with SPEC) users are moving one motor and visualizing more than one counter together. Having multiple signals would allow the user to retrieve that information at the cost of a single click instead of selecting what they want as x axis and at least one click for each of the associated signals. The difference in user experience is incomparable. Users and scientists of our beamlines moving to HDF5 find tedious to have to interact so much.

To summarize with the SPEC analogy, in our view the default plot of an entry associated to a measurement should be able to visualize one of the plots the user was seeing when acquiring the data. If the user was watching multiple counters on a plot, that goal cannot be achieved with the current definition of NXdata and could be solved by implemented the requested feature. You have everything on NeXus to allow that (NXentry at default set to the default NXdata group) except the capability to have multiple signals.


Before we can make meaningful comments, you have to give an explicit
example.

I said we would do it for the XRF fit case and that I expected to have for next week.

In the mean time please think about what the default NXdata plot associated to an entry should be when the user was simultaneously visualizing several counters at acquisition time.

You keep on denying that you mean what both
Ben and I have inferred you to mean, but until we see your example,
it’s impossible to tell.

Sorry but I have to quote yourself and my answer that I thought it could not be clearer:

RAY: but I don't think it is necessary for NeXus to then specify all other possible non-default plots.

ARMANDO: Nobody is asking for it.

As I said, next week there will be the example for a fit output but to have multiple signals is desirable for more than just the fit: exactly the use case of the default plot.

Best regards,

Armando
_______________________________________________
NeXus mailing list
NeXus at nexusformat.org<mailto:NeXus at nexusformat.org>
http://lists.nexusformat.org/mailman/listinfo/nexus

_______________________________________________
NeXus mailing list
NeXus at nexusformat.org<mailto:NeXus at nexusformat.org>
http://lists.nexusformat.org/mailman/listinfo/nexus



--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nexusformat.org/pipermail/nexus/attachments/20180112/749ab765/attachment-0001.html>