[Nexus-developers] NXTranslate: freddy's and ray's last post

[Mark Koennecke]

High,

Freddy wrote:

>I did wonder if nxconvert input files would look better if all scripting
>commands had 
><script></script> around them - that would allow you to embed real data and
>not
>need to use "writeText" commands etc. each time. It would also allow you the
>option
>to use multiple scripting languages or interpreters by passing the
>interpreter reference as an attribute of <script>, or also to do things like
>

This might be a good idea. Especially if we really implement support for
some popular scripting languages.

>A combined nxtranslate/nxconvert scheme where you can reference built-in
>capabilities from something like NXtranslate, but with the option to
>accessing
>external scripting, is probably best. Script could be put in the "data"
>area, and the "mime type" would tell how it should be treated. Unless each
>script was going to be executed by spawning a new process, Nxtranslate 
>would need to know a bit about scripting so it could create
>the initial interpreter and pre-set any required variables. Though spawning
>an
>external process is slower, it does remove any script tie ins - all
>
  
With external scripting, I see a real problem. We might need to move 
megabytes worth of data around. External scripting requires us to parse 
that data from a pipe or whatever before we plug it into the NeXus file.
This is going to be complicated and inefficient.
 
>In both schemes the final file structure has to be specified fully in the
>translation
>file - you have commands to insert data at a given point, but not new groups
>or to
>loop over groups. If you were moving <data1> <data2> ... <datan> from one
>file 
>to another you would have to specify them each individually in the
>translation file, and may need multiple translation files if n varied for
>your
>instruments. However to get round this may mean the translation file looks
>too "script like" and would also require extra parsing and interpreting.

Freddy raises a real problem here. I forgot about it as my original 
dictionary based suggestion solved the problem: data would only be
copied if an alias exists both in the source and target dictionary. The test 
for the availability of data would also be easy.

This is not easily addressed in both NXtranslate (tagged or scripted) 
versions. We cannot delete datasets in HDF4 files. What we have to do is to 
create a dataset only if a precondition (the existence of a path in a 
source file) is met. I still think about this. One ugly solution would be to
have two scripts:
<testscript interpreter="tcl>
  checkPath /entry1/TRICS/title
</testscript>
<script interpreter="tcl">
  copyPath /entry1/TRICS/title
</script> 
Another solution would be to associate any path with a type, i.e:
  nx:///entry1/TRICS/title
and have a tag named depends for the dataset in the translation files 
which holds a komma separated list of source paths this particular data
item depends upon (There could be more then one!). Example:
<title depends="nx://entry1/TRICS/title">
  copyPath nx://entry1/TRICS/title
</title>


Ray wrote:

>Here's the scenario.  The data acquisition system is set up so that, when a
>run starts, all the metadata concerning the run is placed into a standard
>conforming NeXus file, either XML or binary.  However, the data tags are
>left empty but the NXtranslate attributes, which point to the library that
>reads data from electronic memory, are added instead.
>
>When a data analysis program wants to access the data, there are two
>approaches.
>
>1) Run the translation utility to convert the file to a regular NeXus file
>with the NXtranslate attributes replaced by real data.
>
>2) In the longer term, add a thin GetData layer to the NeXus API so that
>external libraries can, optionally, be linked through it to the API.  Data
>analysis programs, on the instrument computer, would link to NeXus API
>containing this external data interface, so that a NXgetdata call would use
>the NXtranslate attributes to get the external data directly.
>
>With the second scenario, the same data analysis programs can work both on
>regular NeXus files and on files where the data is external.  All that would
>be necessary would be to link to the external library.  In some systems,
>this linking could even be done dynamically at run time.

This is also possible with the scripted solution. Adding livedata attributes
to NeXus datasets or pulling live data from somewhere can be done with a
script as well as with any other solution.


	  Best Regards,

		Mark Koennecke