[NeXus-tech] axes for multidimensional data

[Pete Jemian]

this is a copy

On 2/22/2011 1:46 AM, Mark Koennecke wrote:
> Tobias,
>
> Tobias.Richter at diamond.ac.uk wrote:
>>
>> This is to further the discussion we started on the last teleconference.
>>
>>
>>
>> We've got a problem defining useful axes for multidimensional data (or
>> scans).
>>
>>
>>
>> While I've got an idea towards a solution (see below) there might be
>> better ways to doing this. Pete J has indicated he is aware of the
>> problem as well and hasn't found a good solution either.
>>
>>
>>
>> In the 1 dimensional world everything is quite straight forward:
>>
>>
>>
>> data:NXdata
>>
>> data[n]
>>
>> @signal=1
>>
>> x[n]
>>
>> @axis=1
>>
>>
>>
>> For perfect equipment and independent axis it even works for 2D data:
>>
>>
>>
>> data:NXdata
>>
>> data[nx,ny]
>>
>> @signal=1
>>
>> x[nx]
>>
>> @axis=1
>>
>> y[ny]
>>
>> @axis=2
>>
>>
>>
>> This assumes that x recorded as x[i] is identical for every point [i,j].
>>
>> But of course when recording readback values of real motors you will
>> actually receive an array x[nx,ny] because the motor positions are not
>> perfectly reproducible every time you try to visit the same position.
>>
>> In cases were x and y are not independent this scheme does not work at
>> all.
>>
>>
>>
>> With the requirement being to record true values we end up with this:
>>
>>
>>
>> data:NXdata
>>
>> data[nx,ny]
>>
>> @signal=1
>>
>> x[nx,ny]
>>
>> y[nx,ny]
>>
>>
>>
>> The problem is that all information on the axes is lost. There is no
>> way of telling which motor you tried to drive along each of the
>> dimensions.
>>
>> By the way of guessing you can still match up positions and detector
>> readout. For multidimensional detectors (images, etc.) this might not
>> always be easy to guess. Consider e.g.:
>>
>>
>>
>> data:NXdata
>>
>> data[512,512,512,512]
>>
>> @signal=1
>>
>> x[512,512]
>>
>> y[512,512]
>>
>>
>>
>> What dimensions do you need to match up here?
>>
>>
>>
>> The 'recommended' way of unrolling the dimensions:
>>
>>
>>
>> data:NXdata
>>
>> data[nx*ny]
>>
>> @signal=1
>>
>> x[nx*ny]
>>
>> @axis=1
>>
>> y[nx*ny]
>>
>> @axis=1
>>
>>
>>
>> Has the same disadvantage: You don't know what you were driving to
>> arrive at each point. But you also loose the efficient display of the
>> data by slicing. So this is clearly not an option.
>>
>>
>>
>> So the questions are:
>>
>> * How do we annotate that x and y provide the
>>
>> position information?
>>
>> * How do we provide the information on the intended
>>
>> motion along the axes? This would allow at least an
>>
>> axis title (but no ticks and numbers) to be added
>>
>> to a 2D plot of "data" .
>>
>>
>>
>> There are some problems with this but this idea should illustrate what
>> I have in mind:
>>
>>
>>
>> data:NXdata
>>
>> data[nx,ny]
>>
>> @signal=1
>>
>> x[nx,ny]
>>
>> @axis=1,2
>>
>> @label=1
>>
>> y[nx,ny]
>>
>> @axis=1,2
>>
>> @label=2
>>
>>
>>
>> The @axis attributes associate the dimensions of detector and
>> positioner data.
>>
>> The @label allows a labelling of the dimension in a 2D plot of the
>> data. And when you slice along that dimension it gives a preference
>> for the axis to use in the 1D plot.
>>
>>
>>
>> In most cases @label would also imply that the positioner was moved
>> actively along that dimension in a scan. But the may not always be the
>> case. So there might be a benefit in having another indicating
>> attribute like @drive. But that's an aside and we should focus on the
>> main problem first.
>>
>>
>>
>> Let's discuss this here and we can wrap up in the next conference call.
>>
>>
>>
>> Regards,
>>
>>
>>
>> Tobias
>>
>>
> I am a little confused: up to now we had discussed this problem in the
> context of a scan in 2D or more. Not as
> a general thing.
>
> Then in the context of scans in 2D the problem has come up at PSI too. I
> will add another two ideas towards a solution
> to the mess. The first is to use an additional data attribute which
> describes what has been meant:
>
> data:NXdata
> data[nx*ny]
> @signal=1
> @scan_intent=nx:ny
> x[nx*ny]
> @axis=1
> @primary=1
> y[nx*ny]
> @axis=1
>
>
> The other is actually Ben Watts(SLS) idea: have an additional integer
> index array which tells which index
> in the long array maps onto a position on a regular grid:
>
> data:NXdata
> data[nx*ny]
> @signal=1
> x[nx*ny]
> @axis=1
> @primary=1
> y[nx*ny]
> @axis=1
> index[nx,ny]
> @axes=xval,yval
> xval[nx]
> yval[ny]
>
> Names still need to be discussed. To make the scheme extra clear an
> example for a 3x3 scan:
> data:NXdata
> data[9]
> @signal=1
> x[9]
> @axis=1
> @primary=1
> y[9]
> @axis=1
> index[3,3]
> @axes=xval,yval
> xval[3]
> yval[3]
>
> with index being:
>
> 1 2 3
> 4 5 6
> 7 8 9
>
> giving the index into the data array which allows to retrieve the counts
> and the actual positions from
> x and y.
>
>
>
> Best Regards,
>
> Mark
>
>
>>
>>
>> --
>>
>> This e-mail and any attachments may contain confidential, copyright
>> and or privileged material, and are for the use of the intended
>> addressee only. If you are not the intended addressee or an authorised
>> recipient of the addressee please notify us of receipt by returning
>> the e-mail and do not use, copy, retain, distribute or disclose the
>> information in or attached to the e-mail.
>> Any opinions expressed within this e-mail are those of the individual
>> and not necessarily of Diamond Light Source Ltd.
>> Diamond Light Source Ltd. cannot guarantee that this e-mail or any
>> attachments are free from viruses and we cannot accept liability for
>> any damage which you may sustain as a result of software viruses which
>> may be transmitted in or with the message.
>> Diamond Light Source Limited (company no. 4375679). Registered in
>> England and Wales with its registered office at Diamond House, Harwell
>> Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United
>> Kingdom
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> NeXus-tech mailing list
>> NeXus-tech at nexusformat.org
>> http://lists.nexusformat.org/mailman/listinfo/nexus-tech
>
> _______________________________________________
> NeXus-tech mailing list
> NeXus-tech at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus-tech