[Nexus] NeXus update

[V. Armando Sole]

On 21.01.2015 18:47, Pete Jemian wrote:
> On 1/21/2015 11:25 AM, V. Armando Sole wrote:
>> To store x and y as (n_rows, ncolumns) with the product n_rows *
>> n_columns equal to n_points is *not* convenient.
> 
> This is exactly the case for multi-dimensional data acquired with the
> EPICS "sscan" record and stored in APS "MDA" data files.  We have many
> beam lines at the APS using this.
> 
> In many cases, saving x as float[n_rows] is just fine.

I know, but in that case what one should use are HDF5 dimension scales. 
They are there for that purpose. And I am not making my life easier by 
writing that: I do not support them yet.

>  In some cases,
> x is from a readback device with jitter or drift.  That's when x must
> be stored as float[n_rows * ncolumns].  It's a real nuisance to store
> this and visualize, but that is the real-world situation.
> 
> Prior to December 2014, NeXus had no tools to describe this structure
> for visualization.

Which one, a) x as float(nrows * ncolumns) or b) x as float(n_rows) or 
float(n_columns)?

I have been able to represent that for years without needing any 
additional attributes. So, perhaps NeXus had no official convention, but 
the tools were there although not labeled "NeXus".

The representation can be made not thanks to the axes but to the signal 
dimensions: If the user has an array of shape (100, 200, 1024) and tells 
those are spectra (or there is an attribute telling those are spectra) 
it is not difficult at all to visualize that as a map, without even 
asking for the axes coordinates.

However, if one is *directly acquiring* that map, (n_points, 1024) is 
the safe way to go. By the way, SPEC will not be able to save images in 
HDF5 format as (n_points_x, n_points_y, n_rows, n_columns) but as 
(n_points, n_rows, n_columns). So, software willing to visualize SPEC 
mesh scans will have to anyways deal with the generic case.

>   NeXus should make it possible to represent this
> type of data but it should not be required for all data to use this
> structure.

Sorry Peter, but up to now NeXus has established a lot conventions but 
has not "make possible" to do much with them yet. I am just asking to 
avoid making things more difficult to those actually making possible to 
deal with the data. It does not serve to anything to add more and more 
specifications without providing software able to follow them.

Armando