[Nexus] Nexpy

Wed Jun 14 16:47:58 BST 2017

The square brackets are a problem because the target attribute was written as a NumPy string array of shape (1,), which is not then converted to a scalar string.

So in NeXpy, I would normally write:

>>> w1.entry.data=[1,2,3]
>>> w1.entry.data.target=b'123'
>>> w1.nxfile['/entry/data'].attrs['target']
‘123'

or even:

>>> w1.entry.data.target2=np.array('123', dtype=np.string_)
>>> w1.nxfile['/entry/data'].attrs['target2']
b’123'

But I get what your file contains if I write:

>>> w1.entry.data.target3=np.array(['123'], dtype=np.string_)
>>> w1.nxfile['/entry/data'].attrs['target3']
array([b'123'],
      dtype='|S3')

This must be how you have written your file:

>>> nxtree['633777'].nxfile['/entry1/instrument/ic1monitor/ic1monitor'].attrs['target']
array([b'/entry1/instrument/ic1monitor/ic1monitor'],
      dtype='|S41')

This is not being treated as a scalar of shape (), but an array of shape (1,). That means that NeXpy doesn’t think the parent’s path is the same as the target and treats the parent as a link. If you have a lot of files written this way, then I can probably fix this by converting all shape (1,) arrays to scalars.

This is one reason why I would recommend using the nexusformat package to read and write NeXus files. Of course, this assumes you are using Python. Obviously, it the file was written in Java, you are on your own.

With best regards,
Ray

On Jun 14, 2017, at 8:43 AM, steve.collins at diamond.ac.uk<mailto:steve.collins at diamond.ac.uk> wrote:

Thanks Ray,
Aha – that makes sense.
It looks like there might be something wrong with my nxs file (attached). In my example Python can’t do a type conversion to the numbers, but in the chopper example it works fine (see below). We seem to have some extra square brackets – I don’t know if that is significant.
Cheers,
Steve

## Diamond example

n.entry1.default.ic1monitor
Out[204]: NXlink('['/entry1/instrument/ic1monitor/ic1monitor']')

n.entry1.default.ic1monitor+0
Traceback (most recent call last):

  File "<ipython-input-205-336afb990d12>", line 1, in <module>
    n.entry1.default.ic1monitor+0

  File "build/bdist.linux-x86_64/egg/nexusformat/nexus/tree.py", line 2259, in __add__
    return NXfield(value=self.nxdata+other, name=self.nxname,

TypeError: unsupported operand type(s) for +: 'NoneType' and 'int

## chopper example

c.entry.data.polar_angle
Out[202]: NXlink('/entry/instrument/detector/polar_angle')

c.entry.data.polar_angle+0
Out[203]:
NXfield(array([  -7.19999981,   -6.5999999 ,   -6.        , ...,  116.3999939 ,
        116.99999237,  117.59999084], dtype=float32))

From: NeXus [mailto:nexus-bounces at nexusformat.org] On Behalf Of Osborn, Raymond
Sent: 14 June 2017 14:13
To: Discussion forum for the NeXus data format <nexus at nexusformat.org<mailto:nexus at nexusformat.org>>
Subject: Re: [Nexus] Nexpy

Steve,
The NeXus format has, since its inception, had the concept that there is a parent object for each link, defined by the attribute ‘target’, even though linked data are stored in HDF5 files as hard links. NeXpy, or rather the underlying nexusformat API, is designed to make that association clear, whereas h5py makes no such distinction. I’m not sure if there is even a way for h5py to tell what groups an object has been added to without checking object ids.

If you have a recent version of NeXpy installed, you can open the example file, ‘chopper.nxs’, from the Help menu. I will use it to discuss why I believe it handles links appropriately. In ‘chopper.nxs’, the data group contains two links. Here are edited selections from the file tree:

entry:NXentry
  data:NXdata
    polar_angle -> /entry/instrument/detector/polar_angle
  instrument:NXinstrument
    detector:NXdetector
      polar_angle = float32(148)
        @target = '/entry/instrument/detector/polar_angle'

So ‘/entry/data/polar_angle’ and ‘/entry/instrument/detector/polar_angle’ are both paths to the same HDF5 object, but the parent object is deemed by the NeXus format to be the latter, as specified by its target attribute. NeXpy chooses to display the other instance, in ‘/entry/data’ as a link to make this clear.

>>> chopper.entry.data.polar_angle
NXlink('/entry/instrument/detector/polar_angle')
>>> chopper.entry.instrument.detector.polar_angle
NXfield(array([  -7.19999981,   -6.5999999 ,   -6.        , ...,  116.3999939 ,
        116.99999237,  117.59999084], dtype=float32))

If you want to know what the parent object is, you can use the NXlink’s ‘nxlink’ attribute, which returns the parent.

>>> chopper.entry.data.polar_angle.nxlink
NXfield(array([  -7.19999981,   -6.5999999 ,   -6.        , ...,  116.3999939 ,
        116.99999237,  117.59999084], dtype=float32))

Within the NeXpy GUI, you can right-click on the link, which has a chain link icon by it, to display the parent. There is no way to do this in a generic HDF5 viewer.

In practice, the fact that ‘/entry/data/polar_angle’ is an NXlink object should make no difference in your use of the data. For example, you can check its dtype or shape, and use fields in arithmetic expressions on either the parent or the link.

>>> chopper.entry.data.polar_angle + 100.0
NXfield(array([  92.80000305,   93.40000153,   94.        , ...,  216.3999939 ,
        217.        ,  217.59999084], dtype=float32))
>>> chopper.entry.instrument.detector.polar_angle + 100.0
NXfield(array([  92.80000305,   93.40000153,   94.        , ...,  216.3999939 ,
        217.        ,  217.59999084], dtype=float32))

I think this is the right way to handle links. In h5py, you could update a dataset’s value without knowing that it has the side effect of changing its value in another group. In NeXpy, the existence of the link is clear but it shouldn’t impede your use of its values or attributes. I would welcome further comments if you think these are not the right calls.

With best regards,
Ray

On Jun 14, 2017, at 6:13 AM, steve.collins at diamond.ac.uk<mailto:steve.collins at diamond.ac.uk> wrote:

Hello,

I’d like to subscribe to this email list. I’m not sure if it is appropriate to ask a technical question here, but here is one anyway (I hope it’s OK):

I have a question about the Python Nexusformat/NeXpy nxload method.

I seem to have problems reading data entries that have been defined using links. That is, the method returns an NXlink object rather than the actual data:

n.entry1.default.ic1monitor
Out[143]: NXlink('['/entry1/instrument/ic1monitor/ic1monitor']')

If I use h5py directly then I get the dataset:

h['/entry1/default/ic1monitor']
Out[144]: <HDF5 dataset "ic1monitor": shape (10001,), type "<f8">

Is this expected or am I doing something wrong? I read in the manual that all data are specified via links so there is no distinction between the 'original' data and a link. Is that right?

Thank you for your help.

Regards,
Steve Collins (Diamond Light Source)

P.S. I can send you the data files and Python code of course, if that helps.

--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

<633777.nxs>_______________________________________________
NeXus mailing list
NeXus at nexusformat.org<mailto:NeXus at nexusformat.org>
http://lists.nexusformat.org/mailman/listinfo/nexus

--
Ray Osborn, Senior Scientist
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov<mailto:ROsborn at anl.gov>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nexusformat.org/pipermail/nexus/attachments/20170614/b40845ff/attachment-0001.html>