Summary of workshop

[Ray Osborn]

Firstly, I'd like to thank everyone for all the hard work you did in
preparing for the workshop, and for the enthusiasm with which you discussed
everything.  I think it was very productive, and I hope it will help to
consolidate the NeXus project.

I have put my original presentation on the NeXus FTP site (the new one) in
<ftp://ftp.neutron.anl.gov/NeXus/NeXus2000.ppt>.  The last five pages are
probably the most interesting as they include the summary of our
conclusions.  I've appended that summary to the end of this message.

Here are the actions from the meeting.

1) Mark Koennecke: add unlimited dimensions and data flushing to current
API.

2) Nick Maliszewskyj: Start conversion of Fortran 90 utility API to C

3) Mark Koennecke: Recruit post-doc under SCANS project to start writing the
HDF5 version of NAPI.

4) Mark Koennecke: Write Java API with a single class of NXfile.

5) Chris Moreton-Smith: Write draft proposal for defining NeXus files in
XML.

6) Mark Koennecke: When the XML proposal is ready, convert NXdict to use XML
as the dictionary

7) Nick Maliszewskyj: Prepare a configure script for NeXus installation and
look into writing RPM's

8) Przemek Klosowski: Prepare a proposal for a message passing system
between NeXus files and applications, e.g. based on CORBA.

9) Ray Osborn: Bundle web pages as help files for the existing distribution

10) Ray Osborn: Organize instrument sub-committees to prepare instrument
definitions for NeXus workshop.

11) Joern Beckmann: Provide specifications for a NeXus browser.

Let me know if you disagree with any of them, or if I've used the wrong
terminology.  I will combine the workshop summary and actions into a Word
document for general distribution after you've had a chance to look them
over.

Thanks again, and see you at PSI if not before,
Ray

Workshop Summary
================

Review of API
=============
* The current core API is largely complete.
* We need to add ASAP support for unlimited dimensions and a mechanism for
automatic flushing of cached data.
* There is no need for an NXupdate since the only data that can¹t be easily
overwritten are character strings.
* Character strings to be used as history records should have their size
defined as ³unlimited².  Use <lf> as a delimiter.
* There may be a need for NXdelete; this will wait for the HDF5 version.
* We should change the method of defining axes; their names will be stored
as an attribute of the multidimensional data, rather than as an attribute of
the axis SDS.  Both methods should be supported until we migrate the format
to HDF5. 
* There is a need for a higher level API, reproducing the functionality of
the existing Fortran 90 utility API as far as possible in C.

Migration to HDF5
=================
* HDF5 has all the functionality that we require (and more).
* NeXus will start using HDF5 within the next two years.
* The goal is to produce a working version of an HDF5 NAPI by next summer
for people to beta-test.
* The decision of when to make this version official will depend on HDF5
support on critical platforms (though VMS support is uncertain), and on the
existence of wider third-party support.
* We can ensure backward compatibility, by writing the top level of NAPI as
a wrapper with a compile-time switch to regulate which libraries are linked.
* We will provide conversion utilities to rewrite HDF4 files as HDF5.
* We will look into the possibility of using magic files to allow programs
to dynamically link the HDF4 or HDF5 versions.
* There is no need for a change to the core API.  The NXhandle and NXlink
structs will be different.  The tag/ref pairs stored in the NXlink struct
will be replaced by an index pointing to the linked data¹s path.
* Classes have to be stored as group attributes.
* There may be a problem with NXcompress since HDF5 requires the compression
method to be set when the data is created.   It is possible that this can be
solved by caching NXmakedata requests until they are written, but this may
lead to other problems.  This will be investigated.

Java API
========
* There is a need for a Java version of the API.
* Initially, this will be a direct translation of the existing core API with
only one class, NXfile.
* In the future, this could be extended to create NXdata, NXsample, Š
classes.
* Although HDF has provided its own Java wrapper, it would be better for
NeXus to have its own wrapper calling NAPI directly.  This simplifies
distribution and makes future upgrades easier to maintain.
* In the short term, pure Java applications and applets can access NeXus
files through the NeXus Data Server, although this will need to be extended
to allow data output.

XML
===
* We will define the contents of NeXus files in XML.
* Each instrument type will have its own DTD¹s, although this means that all
NXentry groups will have to correspond to the same DTD.
* There will be a general XML DTD for more generic NeXus files.
* The DTD¹s will be used to validate NeXus files
* The DTD will be defined as global attributes.
* We will use XML Schemas to replace DTD¹s when they become the default.
* A working version of a general NeXus XML file will be produced ASAP.
* We will produce a NeXus to XML conversion utility.
* As a long-term goal, XML will be an alternative to HDF for smaller files.
The API will transparently handle both types of file.
* The NXDict program will be rewritten to use XML files as their data
dictionary.

Installation procedures
=======================
* We will provide improved installation procedures starting with automatic
configure scripts, but progressing to the use of RPM¹s.
* We will provide binary installations for Windows and Macs.
* We will also bundle the web files as HTML and PDF documentation.
* The NeXus data server will become part of the standard distribution.

NeXus Workshop
==============
* We will hold a public NeXus workshop at PSI in conjunction with a SCANS
workshop in early 2001.
* This will define the first version of instrument definitions and set up a
management structure for the NeXus format.

Other issues
============
* NeXus.NET - we will investigate a generalized scheme for exchanging NeXus
data between applications e.g. CORBA, message passing using TCP/IP sockets.
* Encourage development of browsers - consider using stripped-down ISAW for
a generic browser and plotter accessing NeXus files using the NeXus Data
Server.


-- 
Dr Ray Osborn                Tel: +1 (630) 252-9011
Materials Science Division   Fax: +1 (630) 252-7777
Argonne National Laboratory  E-mail: ROsborn at anl.gov
Argonne, IL 60439-4845