[Nexus] pixel_mask documentation

Tobias.Richter at diamond.ac.uk Tobias.Richter at diamond.ac.uk
Thu Mar 6 15:55:54 GMT 2014


Hi all,

I have no problem with changing that. I do not believe that bit was in use elsewhere.
Hence I would not mark it as reserved, but maybe deprecated (for now).

Just yesterday we were looking at that bit mask and thought that maybe 32 bits is over the top anyway. And in fact the documentation is wrong to specify using (mask & 0x00FF) as mask because that excludes the user specified bit. When this is under revision we could move bit 8 to 7 (to fix the documentation). And possibly limit the number of bits to 16. That would still leave one bit for hard completely unusable pixels types and 7 (or 6 for signed type users) for soft problems.

Or to you see value in the large number of bits and we just change the documentation to (mask & 0x0000FFFF)?

Regards,

Tobias

________________________________________
From: yayahjb [yayahjb at gmail.com]
Sent: 06 March 2014 14:51
To: Michael Rissi
Cc: Richter, Tobias (DLSLtd,RAL,LSCI); Marcus Mueller; nexus at nexusformat.org
Subject: Re: [Nexus] pixel_mask documentation

I think this is a sensible change, but I would suggest that, in an
abundance of
caution we also clarify with

31 -- reserved -- deprecated former virtual pixel mask bit, not to be
used in new code.

That will help to ensure that nobody tries to reuse it for something else.

On 3/6/14 9:44 AM, Michael Rissi wrote:
> Dear Tobias,
>
> We had some discussions here that bit 31 for masking virtual pixels is
> a little problematic, as we use for various reasons internally int32
> (instead of unsigned int32) to represent bit masks. We therefore
> wondered if it were possible to use bit 30 to mask virtual pixels:
>
> + 0 gap (pixel with no sensor)
> + 1 dead
> + 2 under responding
> + 3 over responding
> + 4 noisy
> + 5 -undefined-
> + 6 pixel is part of a cluster of problematic pixels (bit set in
> addition to others)
> + 7 -undefined-
> + 8 user defined mask (e.g. around beamstop)
> +
> + 9-29 -undefined-
> +
> + 30 virtual pixel (corner pixel with interpolated value)
> + 31 -undefined-
>
> Is it possible at this point to change the definition? Thanks for your
> understanding!
>
> Best regards,
> Michael
>
>
>
>
> On Wed, Jul 10, 2013 at 4:14 PM, <Tobias.Richter at diamond.ac.uk
> <mailto:Tobias.Richter at diamond.ac.uk>> wrote:
>
>     I have captured that in the NXDL like this for now:
>
>     + <field name="pixel_mask" type="NX_INT32" >
>     + <doc>The pixel mask for the detector.
>     + Contains a bit field for each pixel to signal dead, blind or
>     high or otherwise unwanted or undesirabe pixels.
>     +        They have the following meaning:
>     +
>     + 0 gap (pixel with no sensor)
>     + 1 dead
>     + 2 under responding
>     + 3 over responding
>     + 4 noisy
>     + 5 -undefined-
>     + 6 pixel is part of a cluster of problematic pixels (bit set in
>     addition to others)
>     + 7 -undefined-
>     + 8 user defined mask (e.g. around beamstop)
>     +
>     + 9-30 -undefined-
>     +
>     + 31 virtual pixel (corner pixel with interpolated value)
>     +
>     + The normal data analysis software would not take pixels into
>     account when a bit in (mask & 0x00FF) is set.
>     + Tag bit in the upper two bytes would indicate special pixel
>     properties that normally would not be a sole reason to
>     + reject the intensity value (unless lower bits are set as well of
>     course).
>     + </doc>
>
>     It could be prettier, but if we can all agree on it that's almost
>     better.
>
>     Any comments or corrections welcome.
>
>     Regards,
>
>     Tobias
>
>     ________________________________
>     From: Michael Rissi [michael.rissi at dectris.com
>     <mailto:michael.rissi at dectris.com>]
>     Sent: 10 July 2013 10:29
>     To: Richter, Tobias (DLSLtd,RAL,DIA)
>     Cc: Marcus Mueller; Herbert J. Bernstein; nexus at nexusformat.org
>     <mailto:nexus at nexusformat.org>
>     Subject: Re: [Nexus] pixel_mask documentation
>
>     Hi Tobias,
>     That is a good idea! The first two bytes for critical pixel
>     problems, and the rest to mark special pixels. We can still change
>     that, that is fine!
>
>     Best,
>     Michael
>
>
>     On Wed, Jul 10, 2013 at 11:04 AM, <Tobias.Richter at diamond.ac.uk
>     <mailto:Tobias.Richter at diamond.ac.uk><mailto:Tobias.Richter at diamond.ac.uk
>     <mailto:Tobias.Richter at diamond.ac.uk>>> wrote:
>     Hi Michael,
>
>     Thanks for that clarification. I will add that to the
>     documentation of NXdetector.
>
>     The only slight problem I have is that bit 7 is special in that
>     for the 95% (more 99% I would guess) of the users that is a good
>     pixel and hence it should not be "masked". I mean that's why you
>     Dectris guys go through all the effort of manufacturing them like
>     that, right?
>
>     If other vendors create detectors with other types of special
>     pixels you would have to mask out a strange combination of bits of
>     that bit mask to get a useful detector mask.
>
>     How would everyone feel about:
>
>     * moving this corner information to a separate field?
>     * using (for example) bits 31 downwards in the pixel_mask for
>     usable non non-perfect pixels, so that software would normally use
>     (mask & 0x0000FFFF) and will in all likelihood never need to
>     change even when new types of pixels come up?
>
>     Or is it too late for any changes?
>
>     Kind regards,
>
>     Tobias
>
>     ________________________________
>     From: Michael Rissi [michael.rissi at dectris.com
>     <mailto:michael.rissi at dectris.com><mailto:michael.rissi at dectris.com <mailto:michael.rissi at dectris.com>>]
>     Sent: 10 July 2013 06:48
>     To: Richter, Tobias (DLSLtd,RAL,DIA); Marcus Mueller
>     Cc: Herbert J. Bernstein; nexus at nexusformat.org
>     <mailto:nexus at nexusformat.org><mailto:nexus at nexusformat.org
>     <mailto:nexus at nexusformat.org>>
>     Subject: Re: [Nexus] pixel_mask documentation
>
>     Hey Tobias,
>
>     We intend also to mark the invalid pixels in the images themselves
>     with -1. So only if you want to know what is wrong with a marked
>     pixel, you can check the pixel mask.
>
>     Is 6 or 7 targeted for corner pixels on Eiger/Pilatus?
>
>     7 is for corner pixels. Bit 6 is for pixels that lie in a cluster
>     of bad pixels.
>
>
>     To make it easy for consuming software I would not want any bits
>     to be set for a usable pixel value. 95% would not care about a
>     reason for not using the pixel. Flagging up these special pixels
>     somewhere is useful but maybe not in a "mask". Maybe I am wrong
>     anyway.
>
>     What's "5 non uniform" exactly?  Just that I can make the
>     documentation most meaningful.
>
>     I am sorry, that was still there for historical reasons... We can
>     remove this flag, as we do not really need it anymore.
>
>
>     Bits do come in handy though. It allows us to add:
>
>     8 user specified
>     For pixels that are masked out by the user although they have a
>     physically "correct" reading but for example represent artefacts
>     in the measurement like beamstops. Agreed?
>
>     Yes, definitely!
>
>     Best regards,
>     Michael
>
>
>     --
>     This e-mail and any attachments may contain confidential,
>     copyright and or privileged material, and are for the use of the
>     intended addressee only. If you are not the intended addressee or
>     an authorised recipient of the addressee please notify us of
>     receipt by returning the e-mail and do not use, copy, retain,
>     distribute or disclose the information in or attached to the e-mail.
>     Any opinions expressed within this e-mail are those of the
>     individual and not necessarily of Diamond Light Source Ltd.
>     Diamond Light Source Ltd. cannot guarantee that this e-mail or any
>     attachments are free from viruses and we cannot accept liability
>     for any damage which you may sustain as a result of software
>     viruses which may be transmitted in or with the message.
>     Diamond Light Source Limited (company no. 4375679). Registered in
>     England and Wales with its registered office at Diamond House,
>     Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11
>     0DE, United Kingdom
>
>
>
>
>
>
>
>     --
>     **************************************************
>     Michael Rissi, Ph.D.
>     Product Development
>
>     DECTRIS Ltd.
>     Neuenhoferstr. 107
>     5400 Baden
>     Switzerland
>
>     +41 56 500 2100 general
>     +41 56 500 2101 fax
>     +41 56 500 2125 direct
>     michael.rissi at dectris.com
>     <mailto:michael.rissi at dectris.com><mailto:michael.rissi at dectris.com <mailto:michael.rissi at dectris.com>>
>     www.dectris.com <http://www.dectris.com><http://www.dectris.com>
>     **************************************************
>
>
>     Confidentiality Note
>     This message is intended only for the use of the named recipient(s)
>     and may contain confidential and/or privileged information. If you are
>     not the intended recipient, please contact the sender and delete the
>     message. Any unauthorized use of the information contained in this
>     message is prohibited.
>     **************************************************
>
>     --
>     This e-mail and any attachments may contain confidential,
>     copyright and or privileged material, and are for the use of the
>     intended addressee only. If you are not the intended addressee or
>     an authorised recipient of the addressee please notify us of
>     receipt by returning the e-mail and do not use, copy, retain,
>     distribute or disclose the information in or attached to the e-mail.
>     Any opinions expressed within this e-mail are those of the
>     individual and not necessarily of Diamond Light Source Ltd.
>     Diamond Light Source Ltd. cannot guarantee that this e-mail or any
>     attachments are free from viruses and we cannot accept liability
>     for any damage which you may sustain as a result of software
>     viruses which may be transmitted in or with the message.
>     Diamond Light Source Limited (company no. 4375679). Registered in
>     England and Wales with its registered office at Diamond House,
>     Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11
>     0DE, United Kingdom
>
>
>
>
>
>
>
> --
> **************************************************
> Michael Rissi, Ph.D.
> Product Development
>
> DECTRIS Ltd.
> Neuenhoferstr. 107
> 5400 Baden
> Switzerland
>
> +41 56 500 2100 general
> +41 56 500 2101 fax
> +41 56 500 2125 direct
> michael.rissi at dectris.com <mailto:michael.rissi at dectris.com>
> www.dectris.com <http://www.dectris.com>
> **************************************************
>
>
> Confidentiality Note
> This message is intended only for the use of the named recipient(s)
> and may contain confidential and/or privileged information. If you are
> not the intended recipient, please contact the sender and delete the
> message. Any unauthorized use of the information contained in this
> message is prohibited.
> **************************************************


-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 






More information about the NeXus mailing list