<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Many thanks Ray, and everyone else.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Steve<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> NeXus [mailto:nexus-bounces@nexusformat.org]
<b>On Behalf Of </b>Osborn, Raymond<br>
<b>Sent:</b> 16 June 2017 03:25<br>
<b>To:</b> Discussion forum for the NeXus data format <nexus@nexusformat.org><br>
<b>Subject:</b> Re: [Nexus] Nexpy<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi Steve, <o:p></o:p></p>
<div>
<p class="MsoNormal">I have updated nexusformat package to v0.4.8, which should be available on both PyPI and Anaconda (conda update -c nexpy nexusformat). I have checked that it reads your target attributes correctly.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Although Pete says that your file is technically valid (and I trust him to be right on such things), I think it would cause fewer problems if you could write the attributes as scalar strings. For example, if I open your file in h5py and
read your ‘target’ attribute, I get:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">>>> target = h5_file['/entry1/default/Time'].attrs['target'][()]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">>>> target<br>
array(['/entry1/instrument/Waittime/Time'],<br>
dtype='|S33’)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I can only use this as a string within my Python shell either by converting it to a scalar, which is what nexusformat now does, or accessing the first element of the array:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">>>> np.asscalar(target)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">'/entry1/instrument/Waittime/Time'<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">>>> target[0]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">'/entry1/instrument/Waittime/Time’<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I’m not sure how other languages would handle this, so perhaps this is just a Python issue. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">By the way, NeXus (and the nexusformat package) do allow attributes to contain arrays. If you look at the chopper example, the ‘axes’ attribute of ‘chopper.entry.data' is an array of variable length strings (with a special h5py object dtype),
which can be used in list comprehensions, etc:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">>>> chopper.entry.data.axes<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">array(['polar_angle', 'time_of_flight'], dtype=object)<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">However, size 1 arrays are now interpreted as scalars.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">With best regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Ray<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Jun 15, 2017, at 11:50 AM, <a href="mailto:steve.collins@diamond.ac.uk">
steve.collins@diamond.ac.uk</a> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Great – thanks People,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I’ve tried to follow the thread…</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I understand that (a) my nxs file is valid and (b) the new version of nexusformat will solve the problem.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Cheers!</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Steve</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span class="apple-converted-space"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span></span><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">NeXus
[<a href="mailto:nexus-bounces@nexusformat.org">mailto:nexus-bounces@nexusformat.org</a>]<span class="apple-converted-space"> </span><b>On Behalf Of<span class="apple-converted-space"> </span></b>Osborn, Raymond<br>
<b>Sent:</b><span class="apple-converted-space"> </span>15 June 2017 13:25<br>
<b>To:</b><span class="apple-converted-space"> </span>V. Armando Solé <<a href="mailto:sole@esrf.fr">sole@esrf.fr</a>><br>
<b>Cc:</b><span class="apple-converted-space"> </span>Mita, Charles (DLSLtd,RAL,LSCI) <<a href="mailto:charles.mita@diamond.ac.uk">charles.mita@diamond.ac.uk</a>>; Discussion forum for the NeXus data format <<a href="mailto:nexus@nexusformat.org">nexus@nexusformat.org</a>>;
Sharpe, Chris (DLSLtd,RAL,LSCI) <<a href="mailto:chris.sharpe@diamond.ac.uk">chris.sharpe@diamond.ac.uk</a>><br>
<b>Subject:</b><span class="apple-converted-space"> </span>Re: [Nexus] Nexpy</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Thanks, Armando,<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">That is exactly the aim of the nexusformat package - to give scientists a convenient and, hopefully, reliable tool for reading and writing NeXus files that just works. Speaking as a scientist, we don’t want to worry about whether they should
be using variable-length strings or byte arrays - we just want to have variables we can manipulate in Python and store in our data files when required. We don’t want to worry about whether we have obeyed the standard, but would be delighted if the program
sorted that out for us. That is what nexusformat tries to do, and NeXpy adds other conveniences, like showing visually what items are linked in a file tree. <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Those who spend a lot of time developing software, on the other hand, are welcome to do their own thing. If you are happy writing your own read/write modules using, e.g., h5py, then that’s fine. But please don’t expect scientists to have
to learn h5py, then study all the NeXus rules for writing NXdata groups (not forgetting to add an ’NX_class’ attribute, a signal attribute, axes, …), and then worry about whether they should be using byte arrays or strings. <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">This is a philosophical issue that I have brought up many times at NIAC meetings. The original goal of NeXus was to be a convenience tool for scientists, but it is now largely maintained by developers who are willing to read manuals. I
was a little reluctant about deprecating the NAPI for this reason, but it started to become a barrier to using the more advanced features of HDF5 and few scientists write in C anyway. Python is different - scientists will use it if we give them the right modules
- I have a student with little programming experience who writes standard-conforming NeXus files every day. My suggestion that more people use the nexusformat package, rather than just h5py, is that, as this thread shows, it is helpful to have a wide range
of people helping to debug it so that it doesn’t trip scientists up when they do use it. <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">With best regards,<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Ray<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">On Jun 15, 2017, at 3:21 AM, V. Armando Solé <<a href="mailto:sole@esrf.fr"><span style="color:purple">sole@esrf.fr</span></a>> wrote:<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Hello,<br>
<br>
On 15/06/2017 09:17, Carlos Pascual wrote:<br>
<br>
<br>
<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">And, on the other hand, if the NIAC does bless nexusformat as the preferred<span class="apple-converted-space"> </span><br>
access method for python, I think it should be stated unambiguously in the<span class="apple-converted-space"> </span><br>
manual.<o:p></o:p></p>
</div>
</blockquote>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
Well, at the ESRF Python programmers are using h5py.<br>
<br>
However we recommend scientists to use nexusformat as a simpler tool<br>
because they often work with interactive sessions.<br>
As you see our point of view is that nexusformat is just a convenience<br>
tool on top of h5py being the later the standard (Python) tool to access<br>
HDF5 files.<br>
<br>
What we do not recommend at all is to use the NeXus library from Python<br>
(or from anywhere else).<br>
<br>
The use of NAPI was unambiguously discouraged by the NIAC in past NIAC<br>
meetings in favour of standard tools. However I never understood why<br>
trainings were kept going on around an API that was considered obsolete.<br>
<br>
Best,<br>
<br>
Armando<o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">-- <br>
Ray Osborn, Senior Scientist<br>
Materials Science Division<br>
Argonne National Laboratory<br>
Argonne, IL 60439, USA<br>
Phone: +1 (630) 252-9011<br>
Email: <a href="mailto:ROsborn@anl.gov"><span style="color:purple">ROsborn@anl.gov</span></a></span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"> <o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;font-variant-caps: normal;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">-- <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;font-variant-caps: normal;-webkit-text-stroke-width: 0px;word-spacing:0px">
<span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an
authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.<br>
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.<span class="apple-converted-space"> </span><br>
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.<br>
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom<br>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">_______________________________________________<br>
NeXus mailing list<br>
</span><a href="mailto:NeXus@nexusformat.org"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:purple">NeXus@nexusformat.org</span></a><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
</span><a href="http://lists.nexusformat.org/mailman/listinfo/nexus"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:purple">http://lists.nexusformat.org/mailman/listinfo/nexus</span></a><o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif;color:black">-- <br>
Ray Osborn, Senior Scientist<br>
Materials Science Division<br>
Argonne National Laboratory<br>
Argonne, IL 60439, USA<br>
Phone: +1 (630) 252-9011<br>
Email: <a href="mailto:ROsborn@anl.gov">ROsborn@anl.gov</a><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p align="justify"> </p>
<p align="justify">-- </p>
<p align="justify">This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.<br />Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. <br />Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.<br />Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom<br /> </p></body>
</html>