<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
I have now released new versions of NeXpy, v0.10.10, and nexusformat, v0.4.17, which I believe resolve the issues I mentioned on Friday. I apologize again for rushing the earlier release. If you are interested in testing new versions as they are developed,
please let me know. I will start tagging release candidates on the Github before an official release, and would welcome a group of people from different facilities who could check that they don’t cause any problems with the NeXus files they typically deal
with.
<div class=""><br class="">
</div>
<div class="">Please report any issues on the Github pages.</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">NeXpy home page: <a href="http://nexpy.github.io/nexpy/" class="">http://nexpy.github.io/nexpy/</a></div>
<div class="">NeXpy Github: <a href="https://github.com/nexpy/nexpy" class="">https://github.com/nexpy/nexpy</a></div>
<div class="">NeXpy release notes: <a href="https://github.com/nexpy/nexpy/releases" class="">https://github.com/nexpy/nexpy/releases</a></div>
<div class="">NeXpy issues: <a href="https://github.com/nexpy/nexpy/issues" class="">https://github.com/nexpy/nexpy/issues</a></div>
<div class="">nexusformat issues: <a href="https://github.com/nexpy/nexusformat/issues" class="">https://github.com/nexpy/nexusformat/issues</a></div>
<div class=""><br class="">
</div>
<div class="">Technical Note</div>
<div class="">===========</div>
<div class="">This is a note for those developing Python code using the nexusformat API. </div>
<div class=""> </div>
<div class="">One of the issues with the last version arose because of the difficulty in handling fields and attributes that contain, effectively, the same values, but are stored differently. The NeXus standard defines the recommended contents of various fields
and groups, but doesn’t mandate the precise way in which the content is stored. </div>
<div class=""><br class="">
</div>
<div class="">For example, strings may be stored as fixed-length byte strings or variable-length (unicode) strings. The h5py package returns different values for each, with different dtypes and, sometimes, shapes, e.g., () or (1,), and the returned values are
interpreted differently in Python 2 and 3. If this is to be used in a Python script, such as:</div>
<div class=""><br class="">
</div>
<div class=""> if ’time_of_flight’ in group.attrs[‘axes’]:</div>
<div class=""> …</div>
<div class=""><br class="">
</div>
<div class="">it is necessary to ensure that the there is some consistency in the way they are read from the NeXus file, however they are stored. In the past, the nexusformat API hid some of this conversion, making it difficult to retrieve the original stored
values if required. The latest version resolves this issue by creating a new ’nxvalue’ property for both fields and attributes (i.e., NXfield and NXattr objects), to go alongside the existing ’nxdata’ property. Now, the ’nxdata’ property will always contain
the data returned by h5py in its original form, but the ’nxvalue’ property contains the same data, but with two modifications:</div>
<div class=""><br class="">
</div>
<div class=""> * All strings will be converted to Python strings (’str’ in Python 3, ‘unicode’ in Python 2).</div>
<div class=""> * All string arrays will be converted to lists of strings.</div>
<div class=""> * All size-1 arrays will be converted to scalars (using ‘numpy.asscalar’) </div>
<div class=""><br class="">
</div>
<div class="">As an example:</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">>>> field=NXfield((b'a',b'b',b'c'), dtype='S')</div>
<div class="">>>> field.nxdata</div>
<div class="">array([b'a', b'b', b'c'], </div>
<div class=""> dtype='|S1')</div>
<div class="">>>> field.nxvalue</div>
<div class="">['a', 'b', 'c']</div>
</div>
<div class="">
<div class=""><br class="">
</div>
<div class="">The ‘nxvalue’ property is what is returned when accessing the attribute dictionary, but the ‘nxdata’ property can be accessed using the ‘get’ method of the attribute dictionary. In the file recently posted by Armando Solé:</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">>>> xanes['202.1/plotselect'].attrs.get('auxiliary_signals').nxdata</div>
<div class="">
<div class="">array([b'xmap_nika00', b'xmap_nika01', b'xmap_nika02', b'xmap_nika03',</div>
<div class=""> b'xmap_nika04', b'xmap_nika07'], </div>
<div class=""> dtype='|S11')</div>
</div>
<div class=""><br class="">
</div>
<div class="">>>> xanes['202.1/plotselect'].attrs['auxiliary_signals']</div>
<div class="">['xmap_nika00',</div>
<div class=""> 'xmap_nika01',</div>
<div class=""> 'xmap_nika02',</div>
<div class=""> 'xmap_nika03',</div>
<div class=""> 'xmap_nika04',</div>
<div class=""> 'xmap_nika07']</div>
</div>
<div class=""><br class="">
</div>
<div class="">For numerical arrays, larger than size 1, the ‘nxdata’ and ‘nxvalues’ properties are identical. For many NXfield operations, it is not necessary to access either of these properties directly. For example, the ‘nxdata’ property is the one used
automatically in Numpy functions.</div>
<div class=""><br class="">
</div>
<div class="">If you have any comments or suggested improvements about this, I will be happy to consider them.</div>
<div class=""><br class="">
</div>
<div class="">With best regards,</div>
<div class="">Ray</div>
<div class="">-- <br class="">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Ray Osborn, Senior Scientist<br class="">
Materials Science Division<br class="">
Argonne National Laboratory<br class="">
Argonne, IL 60439, USA<br class="">
Phone: +1 (630) 252-9011<br class="">
Email: <a href="mailto:ROsborn@anl.gov" class="">ROsborn@anl.gov</a></div>
</div>
</div>
<br class="">
</div>
</div>
</div>
</body>
</html>