[Nexus] Release of NeXpy v0.10.10 and nexusformat v0.4.17

[Osborn, Raymond]

I have now released new versions of NeXpy, v0.10.10, and nexusformat, v0.4.17, which I believe resolve the issues I mentioned on Friday. I apologize again for rushing the earlier release. If you are interested in testing new versions as they are developed, please let me know. I will start tagging release candidates on the Github before an official release, and would welcome a group of people from different facilities who could check that they don’t cause any problems with the NeXus files they typically deal with.

Please report any issues on the Github pages.

NeXpy home page: http://nexpy.github.io/nexpy/
NeXpy Github: https://github.com/nexpy/nexpy
NeXpy release notes: https://github.com/nexpy/nexpy/releases
NeXpy issues: https://github.com/nexpy/nexpy/issues
nexusformat issues: https://github.com/nexpy/nexusformat/issues

Technical Note
===========
This is a note for those developing Python code using the nexusformat API.

One of the issues with the last version arose because of the difficulty in handling fields and attributes that contain, effectively, the same values, but are stored differently. The NeXus standard defines the recommended contents of various fields and groups, but doesn’t mandate the precise way in which the content is stored.

For example, strings may be stored as fixed-length byte strings or variable-length (unicode) strings. The h5py package returns different values for each, with different dtypes and, sometimes, shapes, e.g., () or (1,), and the returned values are interpreted differently in Python 2 and 3. If this is to be used in a Python script, such as:

    if ’time_of_flight’ in group.attrs[‘axes’]:
        …

it is necessary to ensure that the there is some consistency in the way they are read from the NeXus file, however they are stored. In the past, the nexusformat API hid some of this conversion, making it difficult to retrieve the original stored values if required. The latest version resolves this issue by creating a new ’nxvalue’ property for both fields and attributes (i.e., NXfield and NXattr objects), to go alongside the existing ’nxdata’ property. Now, the ’nxdata’ property will always contain the data returned by h5py in its original form, but the ’nxvalue’ property contains the same data, but with two modifications:

    * All strings will be converted to Python strings (’str’ in Python 3, ‘unicode’ in Python 2).
    * All string arrays will be converted to lists of strings.
    * All size-1 arrays will be converted to scalars (using ‘numpy.asscalar’)

As an example:

>>> field=NXfield((b'a',b'b',b'c'), dtype='S')
>>> field.nxdata
array([b'a', b'b', b'c'],
      dtype='|S1')
>>> field.nxvalue
['a', 'b', 'c']

The ‘nxvalue’ property is what is returned when accessing the attribute dictionary, but the ‘nxdata’ property can be accessed using the ‘get’ method of the attribute dictionary. In the file recently posted by Armando Solé:

>>> xanes['202.1/plotselect'].attrs.get('auxiliary_signals').nxdata
array([b'xmap_nika00', b'xmap_nika01', b'xmap_nika02', b'xmap_nika03',
       b'xmap_nika04', b'xmap_nika07'],
      dtype='|S11')

>>> xanes['202.1/plotselect'].attrs['auxiliary_signals']
['xmap_nika00',
 'xmap_nika01',
 'xmap_nika02',
 'xmap_nika03',
 'xmap_nika04',
 'xmap_nika07']

For numerical arrays, larger than size 1, the ‘nxdata’ and ‘nxvalues’ properties are identical. For many NXfield operations, it is not necessary to access either of these properties directly. For example, the ‘nxdata’ property is the one used automatically in Numpy functions.

If you have any comments or suggested improvements about this, I will be happy to consider them.

With best regards,
Ray
--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nexusformat.org/pipermail/nexus/attachments/20180130/270fdd56/attachment.html>