[Nexus] NeXus - a solution to what is not the real problem ?

[Wuttke, Joachim]

Sebastian:

> using YAML together with its simple API is not contradicting using
> NeXus. NeXus will only force you to give your YAML file specific entries
> and a specific structure. using your example:
>> puts d['Shortpar']['scan_since']
> the user would have to guess that you call the start time "scan_since".
> if you were using the NeXus names, the user knows that he/she has to
> load "start_time". simpler for him/her, no difference for you.

this little example illustrates perfectly my point: at SPHERES, a scan
consists of several subscans, each stored in a separate file. Each
subscan contains the two parameters scan_since and subscan_since.
Therefore, start_time would be ambiguous.

This shows: you cannot unify a raw data format without unifying all
experimental procedures. Therefore, you will always have different
flavors of raw data formats at different instruments, just as described
by Gerd (http://lists.nexusformat.org/pipermail/nexus/2010/000371.html).

> I think that this technical knowledge must be documented: the numbers
> (distances, units, ...) in the data files; the procedure what to do with
> these numbers in some kind of publication. Otherwise the user relies on
> the instrument scientist to an unhealthy degree. I think that it is the
> duty of every user=scientist that he/she can reproduce every step which
> led to his/her results. This must under no circumstances involve a step
> "then the instrument scientist does some voodoo with the data".

And what about the PerkinElmer spectrometer you use for characterizing
your sample ? There you are relying on a black box. In contrast, every
neutron instrument scientist will be glad to explain you whatever you want
to know about the internal working of his machine; he will let you read the
hardware documentation, and give you the data acquisition code. But that
does not mean we must distribute all that material as part of every raw data set.

Since I know you personally, Sebastian, I can assert that you are maximally
remote from being a typical user. Typical users come with less and
less computing skills, and with less and less interest in the details
of the experimental method. My typical user wants to know: at which
temperature the molecule starts to rattle ?

For my own survival, I therefore need to organize the data flow in a way that
users can perform approximately correct standard analyses without wasting
too much of the real power of the experimental method on the one hand, and
without consuming too much of my own time on the other hand. The question
is: would NeXus, by facilitating collaborative software development, help me
to reach that goal ? From todays debate I got the firm impression that the
answer is negative: Converting software and existing data to NeXus would
cost considerable time; it would make the data and software architecture more
complicated; and for the reason given above, it would not even procure
raw data compatibility.

To be perfectly sincere: effort is not even the main argument. If you offered me
to do the complete conversion, I fear I would still refuse for esthetic reasons.
For my taste, the NeXus documentation looks hopelessly bloated. As one
contributor said today: for my purpose, NeXus would be overkill.

Gutes Nächtle - Joachim

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzende des Aufsichtsrats: MinDir'in Baerbel Brumme-Bothe
Geschaeftsfuehrung: Prof. Dr. Achim Bachem (Vorsitzender),
Dr. Ulrich Krafft (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------