[NeXus-committee] units

[Paul Kienzle]

Hi all.  I won't be able to make it to the meeting this week so I will send
my action items to the group list.

The state of units is a mess.  From a programmer perspective I want to be
able to load a value in the units I need for processing regardless of what
units are stored in the file.  The easiest way to achieve this goal is to
store the values for a field with the same units across institutions.  If
that is not feasible, we at least need to use the same names for the same
units so that code to translate is not impossible to write.

The first step in this process is to list the set of units in use.  Rather
than going through the DTDs by hand I wrote a python script which takes a
list of files and for each unit shows the set of fields which use that
unit.

I'm attaching the script and the output.  See the top of the script for
details on running it.

Note that we have a variety of representations for identical units and
a number of fields with no units.

Also note that NXcrystal uses NXFLOAT rather than NX_FLOAT.

        - Paul

-------------- next part --------------
#!/usr/bin/env python

# This program is public domain

# Extract units from the NeXus DTDs
#
# Run using:
#
#   svn co http://svn.nexusformat.org/definitions/trunk definitions
#   python units.py definitions/*/*.xml
#
# Output is
#
#   unit: component.field component.field ...
#
# Once we agree on a format for units, this needs to turn into a
# function for validating the DTD.
#

from xml.dom.minidom import parse,Node

# Global dictionary of unitname: [component.field, component.field, ...]
index = {}

def _process_field(nxclass, node):
  """Interpret the field stored in the DOM element node"""

  # Get the node attributes of interest
  name = node.nodeName
  type  = node.getAttribute('type')
  units = node.getAttribute('units')

  # ref is how the node will be stored in the resulting index
  ref = nxclass+'.'+name

  # Only report units for the float fields
  # We are not looking for NX_FLOAT since the NXcrystal class uses NXFLOAT
  if "FLOAT" in type:
    if False:
      # Output as data tables for a spreadsheet
      desc  = node.childNodes[0].data.strip()
      print "%s,%s,%s,%s,'%s'"%(nxclass,name,type,units,desc)
    else:
      # Index the results by unit
      if units in index:
        index[units] += [ref]
      else:
        index[units] = [ref]


def _process_dtd(file):
  """Interpret the component stored in file"""
  dom = parse(file)
  nxclass = dom.documentElement.tagName
  for node in dom.documentElement.childNodes:
    if node.nodeType == Node.ELEMENT_NODE:
      _process_field(nxclass, node)

def process(file_list):
  """Build an index table for all the units in a set of NeXus DTD files"""
  
  # Process the files
  for file in file_list:
    _process_dtd(file)

  # Print the resulting dictionary in key order
  units = index.keys()
  units.sort()
  for key in units:
    print "%s:\t %s"%(key,' '.join(index[key]))


if __name__ == "__main__":
  import sys
  process(sys.argv[1:])

-------------- next part --------------
:	 NXattenuator.attenuator_transmission NXbeam.final_wavelength NXbeam.incident_polarization NXbeam.final_polarization NXbending_magnet.critical_energy NXbending_magnet.bending_radius NXcollimator.divergence_x NXcollimator.divergence_y NXcollimator.frequency NXcollimator.blade_thickness NXcollimator.blade_spacing NXcrystal.unit_cell NXcrystal.orientation_matrix NXcrystal.temperature NXdata.variable NXdata.variable_errors NXdata.data NXdata.errors NXdetector.distance NXdetector.polar_angle NXdetector.azimuthal_angle NXdetector.dead_time NXdetector.count_time NXevent_data.pulse_height NXfermi_chopper.wavelength NXfilter.temperature NXfilter.unit_cell NXfilter.orientation_matrix NXfilter.m_value NXfilter.substrate_material NXfilter.substrate_thickness NXfilter.coating_material NXfilter.substrate_roughness NXfilter.coating_roughness NXflipper.flip_turns NXflipper.comp_turns NXflipper.guide_turns NXguide.incident_angle NXguide.bend_angle_x NXguide.bend_angle_y NXguide.m_value NXguide.substrate_material NXguide.substrate_thickness NXguide.coating_material NXguide.substrate_roughness NXguide.coating_roughness NXinsertion_device.power NXinsertion_device.energy NXinsertion_device.bandwidth NXlog.time NXlog.average_value NXlog.average_value_error NXlog.minimum_value NXlog.maximum_value NXlog.duration NXmirror.incident_angle NXmirror.bend_angle_x NXmirror.bend_angle_y NXmirror.m_value NXmirror.substrate_thickness NXmirror.substrate_roughness NXmirror.coating_roughness NXmoderator.distance NXmoderator.temperature NXmonitor.preset NXmonitor.range NXmonitor.integral NXmonitor.efficiency NXmonitor.data NXmonitor.sampled_fraction NXorientation.value NXpositioner.soft_limit_min NXpositioner.soft_limit_max NXsample.temperature NXsample.electric_field NXsample.magnetic_field NXsample.stress_field NXsample.pressure NXsample.unit_cell NXsample.orientation_matrix NXsample.relative_molecular_mass NXsample.volume_fraction NXsample.scattering_length_density NXsample.path_length NXsample.path_length_window NXsample.external_DAC NXtranslation.distances NXvelocity_selector.wavelength_spread
10^-3 meter|10^-2 meter:	 NXdetector.x_pixel_offset NXdetector.y_pixel_offset
10^-6 second|10^-7 second:	 NXdetector.time_of_flight
Angstrom:	 NXcrystal.lattice_parameter
Angstrom^-1:	 NXcrystal.scattering_vector
Angstroms:	 NXbeam.incident_wavelength NXbeam.incident_wavelength_spread NXbeam.final_wavelength_spread NXcrystal.wavelength
Angstroms3:	 NXcrystal.unit_cell_volume NXfilter.unit_cell_volume NXsample.unit_cell_volume
Hz:	 NXsource.frequency
MW:	 NXsource.power
MeV:	 NXsource.voltage
amperes:	 NXflipper.flip_current NXflipper.comp_current NXflipper.guide_current
arc minutes:	 NXcrystal.mosaic_horizontal NXcrystal.mosaic_vertical
barns:	 NXattenuator.scattering_cross_section NXattenuator.absorption_cross_section
bars:	 NXdetector.gas_pressure
cm:	 NXattenuator.thickness NXbeam_stop.size NXbeam_stop.x NXbeam_stop.y NXbeam_stop.distance_to_detector NXdetector.detection_gas_path NXdisk_chopper.pair_separation NXdisk_chopper.radius NXdisk_chopper.slit_height NXdisk_chopper.distance NXfermi_chopper.radius NXfermi_chopper.slit NXfermi_chopper.r_slit NXfermi_chopper.height NXfermi_chopper.width NXflipper.thickness NXmoderator.poison_depth NXvelocity_selector.radius NXvelocity_selector.spwidth NXvelocity_selector.length NXvelocity_selector.height NXvelocity_selector.width
degree:	 NXbeam.incident_beam_divergence NXdisk_chopper.slit_angle NXdisk_chopper.phase NXsample.sample_orientation NXsample.rotation_angle
degrees:	 NXbeam.final_beam_divergence NXcrystal.cut_angle NXcrystal.curvature_horizontal NXcrystal.curvature_vertical NXcrystal.polar_angle NXcrystal.azimuthal_angle NXcrystal.bragg_angle NXinsertion_device.phase NXvelocity_selector.twist NXvelocity_selector.table
g:	 NXsample.mass
g cm-3:	 NXsample.density
g.cm-3:	 NXsample.concentration
m:	 NXattenuator.distance NXbeam.distance NXcrystal.segment_width NXcrystal.segment_height NXcrystal.segment_thickness NXcrystal.segment_gap NXcrystal.segment_columns NXcrystal.segment_rows NXsource.distance
meV:	 NXbeam.incident_energy NXbeam.final_energy NXbeam.energy_transfer
meter:	 NXshape.size
metre:	 NXmonitor.distance
micro.second:	 NXsource.pulse_width
microamps:	 NXsource.current
microsecond:	 NXmonitor.time_of_flight
microseconds:	 NXsource.period
mili*metre:	 NXdetector.x_pixel_size NXdetector.y_pixel_size NXinsertion_device.length
milimetre:	 NXinsertion_device.gap NXinsertion_device.taper
minutes:	 NXcollimator.soller_angle
nm:	 NXdisk_chopper.wavelength_range NXvelocity_selector.wavelength
number:	 NXdetector.data NXdetector.data_error
rpm:	 NXfermi_chopper.rotation_speed NXvelocity_selector.rotation_speed
rpm|hertz:	 NXdisk_chopper.rotation_speed
s-1cm-2:	 NXbeam.flux
second:	 NXmonitor.count_time
steradians:	 NXdetector.solid_angle
{units of logged value}:	 NXlog.value
{units of raw values}:	 NXlog.raw_value
{unit}:	 NXmonochromator.wavelength NXmonochromator.wavelength_error NXmonochromator.energy NXmonochromator.energy_error
{}:	 NXpositioner.value NXpositioner.raw_value NXpositioner.target_value NXpositioner.tolerance NXsensor.high_trip_value NXsensor.low_trip_value NXsensor.value NXsensor.value_deriv1 NXsensor.value_deriv2