NeXus dictionary API extension
Mark.Koennecke at psi.ch
Mark.Koennecke at psi.ch
Tue Jul 15 08:24:52 BST 1997
High there,
while implementing the first NeXus files here at PSI, I found that using
the NeXus API is somewhat hideous and requires a lot of repetetive code
in order to implement the structure. In order to facilitate this I have
designed a dictionary based API on top of the NeXus API. This API is
described in the latex file which you will find below. I intend to start
coding this in the next few days or maybe weeks. However, I would
be very grateful for any comments or suggestions concerning this API
extension from all of you.
Mark Koennecke
% Copyleft (c) 1997 by Mark Koennecke at PSI, Switzerland.
%
%
% This software is distributed in the hope that it will be useful,
% but WITHOUT ANY WARRANTY; without even the implied warranty of
% MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
% GNU General Public License for more details.
%
% You may already have a copy of the GNU General Public License; if
% not, write to the Free Software Foundation, Inc., 675 Mass Ave,
% Cambridge, MA 02139, USA.
%
\documentclass[12pt]{article}
\setlength{\oddsidemargin}{-.1in}
\setlength{\evensidemargin}{0in}
\setlength{\topmargin}{0in}
\addtolength{\topmargin}{-\headheight}
\addtolength{\topmargin}{-\headsep}
\setlength{\textheight}{8.9in}
\setlength{\textwidth}{6.2in}
\setlength{\marginparwidth}{0.5in}
\begin{document}
\title{The NEXUS Dictionary API}
\author{Mark K\"onnecke\\
Labor f\"ur Neutronenstreuung\\
Paul Scherrer Institut\\
CH-5232 Villigen PSI\\
Switzerland\\
Mark.Koennecke at psi.ch \\
}
\maketitle
\vskip.3in
\centerline{\large\bf Abstract}
\vskip.2in
\begin{center}
\parbox{.8\textwidth}{
There is a proposed portable data exchange format for neutron and
X-ray scattering communities, NEXUS (described in a separate
publication). Another document describes an application programmers
interface to NEXUS. This is a base level API which hides many of the
hideous details of the HDF interface from the NeXus programmer. The
present document introduces a higher level application programmers
interface sitting on top of the NeXus API. This API (the NEXDICT-API),
reads all file structure interface from a dictionary data file and creates
the structure automatically from that information. The NEXDICT user only
needs to specify the data to write.
}
\end{center}
\clearpage
\section{Introduction}
There exists a prosal for a portable data exchange format for neutron and
X--ray scattering communities, NeXus. NeXus is fully described
elsewhere$^{1}$. NeXus sits on top of the hierachical data format (HDF) as
defined and specified by the National Center for Supercompter Applications,
NCSA, USA. HDF comes with a library of access functions. On top of the
HDF-library an application programmers interface (API) for NeXus was
defined which hides many of the low level details and ideosyncracies of
the HDF interface form the NeXus programmer. However, writing NeXus files stays
hideous even with this interface due to the amount of repetetive code
required to implement the NeXus structure. Now, repetetive tasks is one
area a computer is good at. So, why not have the computer take care of all
the structure associated with the NeXus format? In order to do this two
components are needed:
\begin{itemize}
\item A language which describes the NeXus file structure to the computer.
This language will be called the NeXus Data Definition Language (NXDDL).
NXDLL might also be used as a tool for discussing and defining NeXus
datastructures.
\item A application programmers interface which works with the NeXus Data
Definition Language.
\end{itemize}
Both of the above will be detailed in this document.
\section{The NeXus Data Definition Language}
The NeXus Data Definition Language(NXDDL) has the purpose to define the structure
and data items in a NeXus file in a form which can be understood by a human
programmer and which can be parsed by the computer in order to create the
structure.
For this a dictionary based aproach will be used. This dictionary
will contain pairs of short aliases for data items and definition strings
which hold the structure information. This dictionary will
be initialised from a data file, the NXDDL-file. Such a dictionary can be
used in the following way: Given an apropriate API function, a NXDICT
programmer needs to specify only the alias and the data to write and
everything else is taken care of by the API: vGroup creation, opening,
SDS definition etc. Another use may involve the creation of definition string
completely or partly at run time which can then be used by an API function
in order to create the structures defined by the definition string. The same
holds for writing as well.
A NXDDL dictionary is preferably initialised from a file.
Such a NXDDL file has to follow these general structure guidelines:
\begin{itemize}
\item All input is in US--ASCII.
\item A \verb+#+ in the first column denotes a comment and will be ignored.
\item A \verb+\+ at the end of the line means that the current text will be
continued with the next non-blanck character for the next line.
\item All other entries follow the form: alias = definition string.
This defines \verb+alias+ as a short form for the definition string after the
equality sign.
\end{itemize}
It might be considered to add a special global vGroup of class NXdict to the
NexUs API which holds the dictionary information within a NeXus file.
The next thing to define is the content of the definition string. A
definition string will have the general form: \\
\centerline{\bf PATH/TerminalSymbol}
This means a definition string will consist of a path specifier which
describes the position of a data item in the vGroup hierarchy and a
terminal symbol which describes the nature of the data item.
The path through the vGroup hierarchy to a data item will be described in a
manner analog to a Unix directory hierarchy. However, NeXus requires two
pieces of data in order to fully qualify a vGroup. This is it's name and
class. Consequently, both name and classname will be given for each vGroup,
separated by a komma. A valid path string then looks like: \\
\begin{verbatim}
/scan1,NXentry/DMC,NXinstrument/big_detector,NXdetector/TerminalSymbol
\end{verbatim}
This translates into: TerminalSymbol in vGroup big\_detector, class
NXdetector, which resides in vGroup DMC of class NXinstrument, which in
turn is situated in the vGroup scan1 of class NXentry.
The terminal symbol in a definition string is used to define the data item
at the end of the definition. NeXus currently supports only three types of
data items at the end of the chain: these are scientific data sets (SDS),
vGroups and links to other data items or vGroups. The terminal symbol for a link
is specified by the keyword \verb+NXLINK+
followed
by a valid alias of another data item or vGroup. For example the terminal
symbol: \\
\centerline{\bf NXLINK counts}
would define a link to the data item which is defined by the alias
\verb+counts+.
A vGroup would be denoted by the keyword VGROUP. By then, the vGroup has
already been defined by the path string. This form of alias is only useful
for the definition of links to vGroups.
A SDS is more involved. The definition of an SDS starts with the keyword
\verb+SDS+. This keyword is followed by option value pairs which define the
details of the SDS. The following option exist:
\begin{itemize}
\item {\bf -rank} defines the rank of the SDS.
\item {\bf -dim \{dim0,dim1,dim2,...., dimn\} } defines the dimensions of the
SDS. Exactly the number of rank numbers defining the dimensions
length is required inside the curly braces.
\item {\bf -type} defines the datatype of the SDS as a string corresponding
to the HDF data types.
\item {\bf -name} Specifies the name of the SDS.
\item {\bf -attr \{name,value\} } defines an attribute. In the curly braces
there must be the name and value of the attribute separated by a komma.
\end{itemize}
If no options are given a default is used. This will be a single floating
point number, as this is the most frequently written data item. As an
example see the definition of a 3d array of 32 bit integers:
\begin{verbatim}
PATHSTRING/SDS -name counts -rank 3 -dim {64,64,712} -type DFNT_INT32 \
-attr {Units,Counts}
\end{verbatim}
\section{The NXDICT--API}
In order to interface with the NeXus dictionary API a set of
API--functions is needed. All functions and data types belonging to
this API start with the letters: NXD. The functions belonging to this API
fall into three groups:
\begin{itemize}
\item Dictionary maintainance functions.
\item Data writing and reading functions.
\item Utility functions.
\end{itemize}
One additional data type is needed for this API:
\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap1}
$\langle$tata {\footnotesize 4a}$\rangle\equiv$
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@ typedef void *NXDdict;@\\
\mbox{}\verb@@$\diamond$
\end{list}
\vspace{-1ex}
\footnotesize\addtolength{\baselineskip}{-1ex}
\begin{list}{}{\setlength{\itemsep}{-\parsep}\setlength{\itemindent}{-\leftmargin}}
\item Macro referenced in scrap 7.
\end{list}
\end{minipage}\\[4ex]
\end{flushleft}
NXDdict will be used as a handle for the dictionary currently in use.
\subsubsection{Dictionary Maintainance Function}
\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap2}
$\langle$dicman {\footnotesize 4b}$\rangle\equiv$
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@ NXDdict NXDinitfromfile(char *filename);@\\
\mbox{}\verb@ NXstatus NXDclose(NXDdict handle, char *filename);@\\
\mbox{}\verb@@\\
\mbox{}\verb@ NXstatus NXDadd(NXDdict handle, char *alias, char *DefString);@\\
\mbox{}\verb@ NXstatus NXDget(NXDdict handle, char *alias, char *pBuffer, int
iBufLen);@\\
\mbox{}\verb@@$\diamond$
\end{list}
\vspace{-1ex}
\footnotesize\addtolength{\baselineskip}{-1ex}
\begin{list}{}{\setlength{\itemsep}{-\parsep}\setlength{\itemindent}{-\leftmargin}}
\item Macro referenced in scrap 7.
\end{list}
\end{minipage}\\[4ex]
\end{flushleft}
{\bf NXDinitfromfile} creates a new NeXus dictionary. If filename is NULL, this
is all that happens. If filename is not NULL, it will be opened and the
dictionary will be initialised from the file specified. The return value
is either 0 for failure or non zero for success.
{\bf NXDclose} deletes and writes a NeXus dictionary. If filename is not NULL,
the dictionary specified by handle is written to the file specified by
filename. In any case the dictionary specified by handle will be deleted.
{\bf NXDadd} adds a new alias -- Definition String pair to the dictionary
specified by handle.
{\bf NXget} retrieves the definition string for the alias specified as
the second parameter from the dictionary handle. The definition string
is copied to pBuffer. Maximum iBufLen characters will be copied.
If a special dictionary vGroup as extemsion to NeXus would be accepted,
two more functions need to be defined which read and write the dictionary
from the NeXus file.
\subsubsection{Data Handling functions}
\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap3}
$\langle$dicdata {\footnotesize 5}$\rangle\equiv$
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@ NXstatus NXDputalias(NXhandle file, NXDdict dict, @\\
\mbox{}\verb@ char *alias, void *pData);@\\
\mbox{}\verb@ NXstatus NXDputdef(NXhandle file, char *pDefString, void
*pData);@\\
\mbox{}\verb@@\\
\mbox{}\verb@@\\
\mbox{}\verb@ NXstatus NXDopenalias(NXhandle file, NXDdict dict, @\\
\mbox{}\verb@ char *alias, void *pData);@\\
\mbox{}\verb@ NXstatus NXDopendef(NXhandle file, char *pDefString, void
*pData);@\\
\mbox{}\verb@@\\
\mbox{}\verb@ NXstatus NXDgetalias(NXhandle file, NXDdict dict, @\\
\mbox{}\verb@ char *alias, void *pData);@\\
\mbox{}\verb@ NXstatus NXDgetdef(NXhandle file, char *pDefString, void
*pData);@\\
\mbox{}\verb@@$\diamond$
\end{list}
\vspace{-1ex}
\footnotesize\addtolength{\baselineskip}{-1ex}
\begin{list}{}{\setlength{\itemsep}{-\parsep}\setlength{\itemindent}{-\leftmargin}}
\item Macro referenced in scrap 7.
\end{list}
\end{minipage}\\[4ex]
\end{flushleft}
The NXDICT data handling functions go in pairs. The version ending in
alias expects an NXDdict and an alias as input. These routines work
out the pass from that. The other version ending on def acts upon
a definition string specified as second parameter. Using this scheme
both full dictionary operation is possible, as well as operation with
program generated definition strings. All routines return the
usual NeXus status returns. All these routines start at the current vGroup
level and return back to it.
NXDputalias, NXDputdef write the data element specified by the alias or
the definition string to the NeXus file specified as first parameter.
pData is a pointer to the data to be written. These routines will check for
the existence of all vGroups required in the path part of the definition
string. If a vGroup is missing it will be created.
NXDopenalias, NXDopendef open the specified data items specified by the
alias or the definition string. Then the usual NeXus functions can be
used to interact with the data. These routines use the same scheme for
creating vGroups on the fly as the put routines above.
NXDgetalias, NXDgetdef read a data item from file. pData MUST point to a
data area large enough to hold the data read. If a vGroup is missing in
the path for one of these routines an error is generated because it is
assumed that the data is present if a program wants to read it.
\subsection{NXCDICT Utility Functions}
This section list a couple of functions which either perform common
tasks on NeXus files or relate
to aspects of error handling and debugging.
\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap4}
$\langle$dicutil {\footnotesize 6}$\rangle\equiv$
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@ NXstatus NXDwriteglobals(NXhandle file, @\\
\mbox{}\verb@ char *filename,@\\
\mbox{}\verb@ char *owner,@\\
\mbox{}\verb@ char *adress,@\\
\mbox{}\verb@ char *phone,@\\
\mbox{}\verb@ char *e-mail,@\\
\mbox{}\verb@ char *place);@\\
\mbox{}\verb@@\\
\mbox{}\verb@ NXstatus NXDmakeentry(NXhandle file, char *pName);@\\
\mbox{}\verb@ @\\
\mbox{}\verb@ NXstatus NXDsetverbosity(int iLevel); @\\
\mbox{}\verb@@$\diamond$
\end{list}
\vspace{-1ex}
\footnotesize\addtolength{\baselineskip}{-1ex}
\begin{list}{}{\setlength{\itemsep}{-\parsep}\setlength{\itemindent}{-\leftmargin}}
\item Macro referenced in scrap 7.
\end{list}
\end{minipage}\\[4ex]
\end{flushleft}
{\bf NXDwriteglobals} writes the global attributes to a newly opened
NeXus file. The parameters should be self explaining. In addition
the file creation date is automatically written.
{\bf NXDmakeentry} creates a new entry vGroup at root level with the
name specified. The group creation date is automatically written.
{\bf NXDsetverbosity} sets a verbosity level. Currently only two are
supported: NXDquiet and NXDalot. If NXDquiet is set (the default) the
NXD routines perform their tasks quietly, ownly complaining about
fatal error conditions. If NXDalot is set, the NXD routines will comment
on any step they do. This facility will help to debug dictionary files
and the software. The output happens through the same scheme which
is used for NeXus error handling, i.e. a configurable error output
function.
\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap5}
\verb@"nxdict.h"@ {\footnotesize 7 }$\equiv$
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@/*--------------------------------------------------------------------
-------@\\
\mbox{}\verb@ NXDICT API header file@\\
\mbox{}\verb@@\\
\mbox{}\verb@ copyleft: Mark Koennecke, March 1997 at LNS,PSI, Switzerland@\\
\mbox{}\verb@@\\
\mbox{}\verb@ No warranties of any kind taken.@\\
\mbox{}\verb at ----------------------------------------------------------------------
------*/@\\
\mbox{}\verb@#ifndef NXDICTAPI@\\
\mbox{}\verb@#define NXDICTAPI@\\
\mbox{}\verb@#include "napi.h" /* make sure, napi is included */@\\
\mbox{}\verb@@\\
\mbox{}\verb@/*-------------------- NXDict data types & defines
----------------------*/@\\
\mbox{}\verb@@$\langle$tata {\footnotesize 4a}$\rangle$\verb@@\\
\mbox{}\verb@#define NXquiet 0@\\
\mbox{}\verb@#define NXalot 1@\\
\mbox{}\verb@/*-------------------- Dictionary Maintainance
----------------------------*/@\\
\mbox{}\verb@@$\langle$dicman {\footnotesize 4b}$\rangle$\verb@@\\
\mbox{}\verb@/*----------------- Dictionary added data transfer
-----------------------*/ @\\
\mbox{}\verb@@$\langle$dicdata {\footnotesize 5}$\rangle$\verb@@\\
\mbox{}\verb@/*-------------------- Utility Functions
--------------------------------*/@\\
\mbox{}\verb@@$\langle$dicutil {\footnotesize 6}$\rangle$\verb@@\\
\mbox{}\verb@#endif@\\
\mbox{}\verb@@$\diamond$
\end{list}
\vspace{-2ex}
\end{minipage}\\[4ex]
\end{flushleft}
\end{document}
--------------------------------------------------------------------------------
Return-Path: <owner-nexus at anpns1.pns.anl.gov>
Received: from dns2.anl.gov by ANPNS1.PNS.ANL.GOV (MX V4.1 AXP) with SMTP; Tue,
15 Jul 1997 02:25:22 CST
Received: from pss201.psi.ch (pss201.psi.ch [129.129.40.201]) by dns2.anl.gov
(8.6.11/8.6.11) with SMTP id CAA18906 for <NEXUS at anl.gov>; Tue, 15
Jul 1997 02:25:15 -0500
Received: from psiclc.psi.ch by pss201.psi.ch; Tue, 15 Jul 97 09:27:40 +0200
Message-ID: <97071509245208 at psiclc.psi.ch>
More information about the NeXus
mailing list