[NeXus-tech] Fwd: Re: follow-up from our discussion today

[Pete R. Jemian]

-------- Original Message --------
Subject: Re: follow-up from our discussion today
Date: Thu, 20 Oct 2011 16:27:49 -0500
From: Chris Jacobsen <chris.j.jacobsen at gmail.com>
To: Jemian Pete <jemian at anl.gov>
CC: De Carlo Francesco <decarlo at aps.anl.gov>,        Saunders Claude 
<saunders at aps.anl.gov>,        Schwarz Nicholas <nschwarz at aps.anl.gov>

Two other points:
1. Access directly to HDF5 allows one to use HDF5 compression.
2. Access directly to HDF5 allows one to use MPI support for parallel 
read/write in HDF5 files.
Unless you want to implement that in a NeXus API, why have a API?

On Oct 20, 2011, at 3:56 PM, Chris Jacobsen wrote:

> Hi Pete -
>
> Please feel free to forward this message to your NeXus code camp people.
>
> I really enjoyed the chance to have a dialogue with the NeXus people today.  I attach my slides.  I was not shy in pointing out what I have perceived to be shortcomings in making use of NeXus, and I fully realize that some of my criticisms are based as much on ignorance as on problems with NeXus.  However, as I said before, I have tried a few times - first when doing the programming myself a few years ago, and more recently in working with Francesco De Carlo, Claude Saunders, and Nicholas Schwartz - and in each instance I have stepped away from NeXus because of problems in getting the API to work on several platforms and languages, and on the sheer length of the documentation making it appear to be hopelessly complex.  As I said, I have figured out netCDF and HDF5 on my own, with no problems in installing libraries, so while one can argue about the percentage I can at least say that I'm not 100% incompetent.
>
> It sounds to me like NeXus is gradually increasing its adoption at facilities where a computer support group plays a strong role in writing the code to save experimental data. Where it has made less inroads is at beamlines where a scientist is bashing out code on a part-time basis.  So how can one make it to their advantage for people like that to adapt NeXus? (Because that's the only way they'll use it - if they see advantages). Here are a few thoughts:
> 1. Make it easy for people to write simple NeXus compliant files containing at least one data cube, with just calls to HDF5 and no need to worry about installation of an API.  You then avoid the problem of people giving up on NeXus because of problems with the API in language X on system Y.
> 2. With respect to 1, also make a clear "quickstart" manual on how to read and write the most basic of NeXus files, and then furthermore provide examples of NeXus files, and the source code used to write them, for several of the more popular techniques along with screen snapshots of what the data should look like.
> 3. Address the technical problems that Claude raised.
>
> One should not have to be part of the NeXus developer club to figure out how to use it for basic data exchange; I still get that vibe from some in the NeXus camp, however.  Make it easy and well documented, and you'll lower barriers to its adoption.
>
> CJ
>
>
> <2011_10_20_nexus_heretics_v5.pdf>
> ------------------
> Chris Jacobsen (cjacobsen at anl.gov, +1-630-252-7960)
> Associate Division Director, APS XSD, Argonne National Lab
> Professor, Physics & Astronomy, Northwestern University
> Chemistry of Life Processes Institute, Northwestern University
> http://xrm.phys.northwestern.edu/~jacobsen/index.html
>