[Nexus] Correct way to specify multiple signals

Pete Jemian jemian at anl.gov
Fri Jan 12 16:34:24 GMT 2018

I'm thinking XRF is a classic case of multiple 2-D charts.  Not 
displayed on top of each other, that's a mess.  A grid of 2-D charts 
each for a different fluorescence wavelength.

On 1/12/2018 8:53 AM, Osborn, Raymond wrote:
> I know I’m being cast as an obstructionist Luddite in this thread, who 
> isn’t able to understand the obvious, but it comes from experience that 
> we make bad decisions when we don’t think through the implications. I am 
> perfectly capable of understanding the desire for multiple signals, but 
> we need to experiment with a number of practical examples before we 
> understand what the consequences of this decision are.
> Here’s one that needs clarification. In Pete Jemian’s SPEC importer, he 
> converts meshes into 2D data, producing an NXdata like this:
> data:NXdata
>   @axes = [‘K’ ‘H']
>   @signal = ‘NaI'
>   @auxiliary = [‘detector’ ‘ic0’ ‘ic1’ ‘ic2’ ic3’]
>   @auxiliary_axes = [[‘K’ ‘H’] [‘K’ ‘H’] [‘K’ ‘H’] [‘K’ ‘H’] [‘K’ ‘H’]]
>   Epoch = float64(16x16)
>   H = float64(16)
>   K = float64(16)
>   L = float64(16x16)
>   NaI = float64(16x16)
>   Ring = float64(16x16)
>   Seconds = float64(16x16)
>   detector = float64(16x16)
>   ic0 = float64(16x16)
>   ic1 = float64(16x16)
>   ic2 = float64(16x16)
>   ic3 = float64(16x16)
> As you can see, I added attributes following what I believe is one of 
> the suggestions for allowing extra signals. Since the data is 2-D, the 
> axes are size-2 arrays, so each auxiliary axis also has to be an array 
> of size-2 arrays matching the axes. (By the way, the latest version of 
> the standard requires extra attributes of the form ‘Two_Theta_indices’ 
> stating what dimension they refer to - it’s not an aspect of the 
> standard that I support, but it would make the above example much more 
> complicated).
> Note that, without the auxiliary attributes, the above is a 
> NeXus-conforming data set, so it already contains multiple signals and 
> multiple axes.
> OK - I think everyone is thinking of 1-D arrays, because it’s difficult 
> to overplot 2-D arrays, but 1D arrays have their difficulties as well.
> data:NXdata
>   @axes = 'Two_Theta'
>   @signal = 'NaI'
>   @auxiliary = [‘detector’ ‘ic0’ ‘ic1’ ‘ic2’ ic3’]
>   @auxiliary_axes = [’Seconds’, ‘Epoch’, ‘Epoch’, ’Two_Theta']
>   Epoch = float64(26)
>   H = float64(26)
>   K = float64(26)
>   L = float64(26)
>   NaI = float64(26)
>   Ring = float64(26)
>   Seconds = float64(26)
>   Two_Theta = float64(26)
>   detector = float64(26)
>   ic0 = float64(26)
>   ic1 = float64(26)
>   ic2 = float64(26)
>   ic3 = float64(26)
> Here I have made an arbitrary choice of which x-axis to plot against 
> which y-axis, but what is a generic NeXus program, like NeXpy, meant to 
> do here? Is it meant to provide an option to plot every possible signal 
> against its axis on the same plot, resulting in six different y-axes and 
> four different x-axes, all of which might need to be arbitrarily scaled 
> against each other (or do we provide the scaling as well?). That is not 
> trivial to do in Matplotlib.
> My point is that I can see why people want multiple signals, but it may 
> be difficult to define them in a way that would allow be unambiguously 
> parsed in the way that a standard demands. I’m not saying it is 
> impossible, but if we are to extend the standard, these need to be 
> thought through.
> Remember that, on your beamlines, there is nothing to stop you adding 
> menu items to your applications that will overplot specific signals that 
> are typically plotted together.
> In the last day, I have provided four different explicit examples to 
> show why this might be complicated.  I am extremely willing to be 
> convinced if you provide your own examples that show why this should be 
> a part of the standard.
> Ray
>> On Jan 12, 2018, at 1:14 AM, GOTZ Andrew <andy.gotz at esrf.fr 
>> <mailto:andy.gotz at esrf.fr>> wrote:
>> Armando,
>> thank you for your patience - your explainations are clear to me! It 
>> is a pity the Nexus community do not understand them
>> Andy
>> On 12/01/2018 06:54, V. Armando Sole wrote:
>>> On 11.01.2018 23:45, Osborn, Raymond wrote:
>>>> But you haven’t explained why you need multiple signals. If it’s
>>>> to store the fit with the raw data, then you will also probably need
>>>> multiple axes, since fitted functions are normally calculated at many
>>>> more points than the raw data. How would you handle that? And do you
>>>> propose to store a copy of the fields within the NXentry raw NXdata
>>>> group?
>>> I have proposed both and explicitly mentioned that auxiliary_signals 
>>> and auxiliary_axes would allow to achieve co-plotting raw_data and 
>>> fitted data with different number of points in my reply to Peter's 
>>> proposal.
>>> You are focused on the fit, but any data treatment generating more 
>>> than one "signal" makes desirable to be able to group them. PyMca and 
>>> NexPy have no problem on visualizing anything, but plenty of 
>>> developers complain about the fat they are "forced" to create a 
>>> separated NXdata for each "signal" if they want the end user to 
>>> inspect the data in a friendly way. The fit case is just so evident 
>>> that should be enough to justify having that feature.
>>> Other simple use case that I aim to have solved at the ESRF is to 
>>> achieve the goal of providing the user with a simple way to browse 
>>> the entries of an HDF5 file generated during the acquisition and 
>>> presenting them what they had at acquisition time. As you know 
>>> sometimes (even with SPEC) users are moving one motor and visualizing 
>>> more than one counter together. Having multiple signals would allow 
>>> the user to retrieve that information at the cost of a single click 
>>> instead of selecting what they want as x axis and at least one click 
>>> for each of the associated signals. The difference in user experience 
>>> is incomparable. Users and scientists of our beamlines moving to HDF5 
>>> find tedious to have to interact so much.
>>> To summarize with the SPEC analogy, in our view the default plot of 
>>> an entry associated to a measurement should be able to visualize one 
>>> of the plots the user was seeing when acquiring the data. If the user 
>>> was watching multiple counters on a plot, that goal cannot be 
>>> achieved with the current definition of NXdata and could be solved by 
>>> implemented the requested feature. You have everything on NeXus to 
>>> allow that (NXentry at default set to the default NXdata group) except 
>>> the capability to have multiple signals.
>>>> Before we can make meaningful comments, you have to give an explicit
>>>> example.
>>> I said we would do it for the XRF fit case and that I expected to 
>>> have for next week.
>>> In the mean time please think about what the default NXdata plot 
>>> associated to an entry should be when the user was simultaneously 
>>> visualizing several counters at acquisition time.
>>>> You keep on denying that you mean what both
>>>> Ben and I have inferred you to mean, but until we see your example,
>>>> it’s impossible to tell.
>>> Sorry but I have to quote yourself and my answer that I thought it 
>>> could not be clearer:
>>> RAY: but I don't think it is necessary for NeXus to then specify all 
>>> other possible non-default plots.
>>> ARMANDO: Nobody is asking for it.
>>> As I said, next week there will be the example for a fit output but 
>>> to have multiple signals is desirable for more than just the fit: 
>>> exactly the use case of the default plot.
>>> Best regards,
>>> Armando
>>> _______________________________________________
>>> NeXus mailing list
>>> NeXus at nexusformat.org <mailto:NeXus at nexusformat.org>
>>> http://lists.nexusformat.org/mailman/listinfo/nexus
>> _______________________________________________
>> NeXus mailing list
>> NeXus at nexusformat.org <mailto:NeXus at nexusformat.org>
>> http://lists.nexusformat.org/mailman/listinfo/nexus
> -- 
> Ray Osborn, Senior Scientist
> Materials Science Division
> Argonne National Laboratory
> Argonne, IL 60439, USA
> Phone: +1 (630) 252-9011
> Email: ROsborn at anl.gov <mailto:ROsborn at anl.gov>
> _______________________________________________
> NeXus mailing list
> NeXus at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus

Pete R. Jemian, Ph.D.                <jemian at anl.gov>
Beam line Controls and Data Acquisition, Group Leader
Advanced Photon Source,   Argonne National Laboratory
Argonne, IL  60439                   630 - 252 - 3189
    Education is the one thing for which people
       are willing to pay yet not receive.

More information about the NeXus mailing list