[Nexus] proposed additions to NXdata for non-linear scaling - to aid cbf interoperability

[Benjamin Watts]

Hi All,
    Perhaps let's take a quick look at what would be required to 
implement and support an NXformula class. The 2010 NIAC minutes mention 
"muParser" 
(http://muparser.beltoforion.de/mup_features.html#idFeatureOverview) for 
the syntax and it could be used to cover high-performance needs in C++, 
C and C#. I expect we should also support implementations in Python and 
Java (at least), which should be doable for the short list of muParser 
built-in functions and telling people needing high performance to use 
muParser instead.

I would propose that NXformula be implemented as a group containing:
* A field named "formula" containing a string representing the formula.
* Links with names corresponding to the parameters of the formula and 
pointing to the appropriate datasets.

E.g.
Scaling:NXformula
   formula="A*sin(B)+C/3.5"
   A:NXlink -> /entry1/counter0/data
   B:NXlink -> /entry1/sample/rotation_angle
   C:NXlink -> /entry1/counter1/data


Does this sound like a "good" way to do it?

Cheers,
Ben


On 19/02/14 14:28, Mark Koennecke wrote:
> Hi,
>
>
> On 02/19/2014 11:15 AM, Wintersberger, Eugen wrote:
>> Hi folks
>>
>> On Wed, 2014-02-19 at 10:11 +0100, Benjamin Watts wrote:
>>> Hi Jonathan,
>>>      This kind of thing has been discussed at previous NIAC meetings.
>>> Your proposal makes for a simple change, but does more to push the
>>> problem further away from yourself than to actually solve it.
>> If I understand Ben correctly what he means is that the code working
>> with the stored data needs to do a lot of work to interpret it
>> correctly.
>>
>>> What we
>>> really need is some "standard" way to present mathematical formulae in
>>> general that is accessible across different programming languages.
>> That's the right way to go.
>>
>>> I
>>> think maybe Eugen Wintersberger had a good suggestion, but I don't
>>> recall any follow-up on it.
>> Hm - I cannot recall that I made a good suggestion on this. However,
>> what I would mot probably do is to use Python syntax to represent math.
>> This has two advantages:
>>
>> 1.) an actual language can immediately execute the code
>> 2.) it might be easy to write a parser for Python code in any other
>> language.
>>
>> In my opinion 2. is the the critical point. Once you start with this you
>> are basically inventing a new scripting language. Consequently, a
>> program which wants to use the data needs to parse this code and takes
>> the appropriate action which is not a trivial task.
>>
>> In addition one should be very careful with such things to not open
>> Pandoras box. I would restrict this to plain math (no loops, no
>> branching, etc.). Otherwise you really invent a new language.
>>
>> What is still open is how to fill the namespace for the code to execute.
>> Where should all the data mentioned in
>>
>> offset  + data**c
>>
>> come from?
>>
>> At the end of the day there is one question we should seriously discuss:
>> how far do we want to go with math in Nexus?
> This is exactly the point why we never got down to anything in this.
> In 2011, Amando was tasked to make a proposal but nothing ever
> came from it. With the manpower and support we do not have, we
> cannot support our own data interpreter, cross platform and good
> performance wise.
>
> IMHO, the wisest solution seems to be to add what is
> absolutely required. Required in the sense to cover the use case where
> you do not have the time to do the conversion on the fly before
> writing the data to disk.
>
> Regards,
>
>             Mark
>
>
>
>
>
>> regards
>>    Eugen
>>
>>
>>
>> _______________________________________________
>> NeXus mailing list
>> NeXus at nexusformat.org
>> http://lists.nexusformat.org/mailman/listinfo/nexus
>
> _______________________________________________
> NeXus mailing list
> NeXus at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus