[NeXus-committee] Fwd: Summary Hangout NeXus format for Pixelated STEM

Tue Apr 16 14:27:01 BST 2019

Dear Dieter,
    Your email did make it to the list. The issue is probably more that 
everyone does NeXus "on the side" and so tends to busy with their "day 
job". If you still need help, then making noise and prodding specific 
people is probably the best way to get the attention you need.
While I don't have any experience with STEM, I am using NeXus for 
scanning transmission X-ray microscopy (STXM). This is probably very 
similar , just using X-rays and scanning the sample, rather than 
scanning an electron beam. You might find it instructive to read the 
article I wrote about applying NeXus to STXM:
https://aip.scitation.org/doi/abs/10.1063/1.4937536
If you think I can be helpful, then let me know and I'll gladly send you 
some example NeXus files and give what advice I can. If I understand 
your scanning question from the mockup NeXus tree, I think you could 
follow the same philosophy as NXstxm:
1. treat the scan position coordinates as detectors in NXinstrument
2. each detector contains data as a list of scan-points, with additional 
dimensions for detectors that return multiple values. For example, a 2D 
area detector with 128x128 pixels would have a data array of 
10000x128x128 for a scan covering a 100x100 point grid, while the 
corresponding position "detectors" would each have a list of 10000 
coordinate values.
3. the NXdata groups simplify the data such that only one value is given 
per scan point and it is assumed that the scan forms a regular grid (and 
it is accepted that this might not be fully correct). For the above 
example, the data would be presented as a 100x100 array - for 
bright-field data, you could simplify the 128x128 images into single 
values by taking the sum of pixel intensities.

This strategy does mean that data gets written multiple times, but HDF5 
compression keeps file sizes manageable and we get a good compromise 
between recording raw values and convenient visualisation.

Best regards,
Ben

On 04/15/2019 02:56 PM, Dieter Weber wrote:
> Dear Pete,
> dear Michael,
> CC Tobias,
>
> about a year ago we started a discussion about building NeXus
> Application Definitions for pixelated STEM. Sadly, I didn't receive any
> response from the mailing list for the message below.
>
> Did it actually get sent or was it "eaten" by a spam filter? Are you and
> the rest of the NeXus community still interested in the topic?
>
> As far as I can remember, the main issue where we needed help was
> expressing beam scanning correctly, which is of course an integral part
> for scanning transmission electron microscopy (STEM).
>
> In case the topic is still relevant for NeXus, what should we do to move
> this project forward? Who else on NeXus side could be interested and
> competent to help set up an Application Definition and extend NeXus
> towards electron microscopy?
>
> With many thanks in advance,
> Dieter
>
>
> -------- Forwarded Message --------
> Subject: Summary Hangout NeXus format for Pixelated STEM
> Date: Wed, 30 May 2018 13:12:11 +0200
> From: Dieter Weber <d.weber at fz-juelich.de>
> To: libertem at googlegroups.com <libertem at googlegroups.com>,
> nexus at nexusformat.org
>
> Dear all,
>
> thank you for the productive call and your contributions! As agreed,
> here's a short summary.
>
> Participants: Magnus Nord (UAntwerpen), Michael Cao (Cornell), Pete
> Jemian (APS), Vadim Migunov, Alex Clausen, Dieter Weber (FZ Jülich)
>
> We developed a mockup NeXus tree with comments and TODO items in a
> shared editor: https://hackmd.io/g6f77f2TRJm_cveiIgNFSQ?both
>
> I've saved the current draft in the LiberTEM GitHub repository as well
> for safekeeping:
> https://github.com/LiberTEM/LiberTEM/blob/master/fileformat/draft%20NeXus%20tree.txt 
>
>
> Next steps:
>
> * Please comment and modify the draft! You can just add comments and
> make changes in the shared editor if you like, or send feedback on the
> mailing list.
> * Continue with Magnus' work to build readers and writers as soon as the
> draft stabilizes
> * Implement in relevant software for testing.
>
> I'm very happy about our progress, thanks again to all contributors, and
> I'm looking forward to continue on this path together!
>
> Greetings,
> Dieter
>
> Appendix: Links from the Hangout chat
>
> http://download.nexusformat.org/doc/html/classes/base_classes/NXsource.html#nxsource 
>
> http://download.nexusformat.org/doc/html/classes/applications/NXstxm.html#nxstxm 
>
> https://github.com/prjemian/spec2nexus/blob/master/src/spec2nexus/eznx.py#L182 
>
> http://download.nexusformat.org/doc/html/utilities.html?highlight=validation#validation 
>
> http://download.nexusformat.org/doc/html/classes/base_classes/NXpositioner.html#nxpositioner 
>
> http://download.nexusformat.org/doc/html/classes/base_classes/NXtransformations.html#nxtransformations 
>
> http://download.nexusformat.org/sphinx/classes/base_classes/NXpositioner.html 
>
> http://cansas-org.github.io/canSAS2012/examples.html#d-image
> Contact at HDF Group regarding chunking: Elena Pourmal
> <epourmal at hdfgroup.org>
> http://download.nexusformat.org/doc/html/datarules.html?highlight=unidata#nexus-data-units 
>
> thumbnail images are a proposal in NeXus that looks like it is
> converging to implementation:
> https://github.com/nexusformat/definitions/issues/381
> NeXus conference call (biweekly)
> https://www.nexusformat.org/Teleconferences.html
>
>
> -- 
> Dr. Dieter WEBER
>
> Peter Grünberg Institute, Microstructure Research (PGI-5)
> Ernst Ruska-Centre for Microscopy and Spectroscopy with Electrons (ER-C)
> Forschungszentrum Jülich
> 52425 Jülich, Germany
>
> Email: d.weber at fz-juelich.de
> Phone: +49 2461 61 85118
>
>
> ------------------------------------------------------------------------------------------------ 
>
> ------------------------------------------------------------------------------------------------ 
>
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> ------------------------------------------------------------------------------------------------ 
>
> ------------------------------------------------------------------------------------------------ 
>
>
>
> _______________________________________________
> NeXus-committee mailing list
> NeXus-committee at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus-committee