[Nexus] how to read files from Eiger?

[Herbert J. Bernstein]

Dear Pete,

  If you are on a system with good dlfnc support and are using a recent
HDF5 library, then use of the plugin filters is transparent (just install
the filters and define the HDF5_PLUGIN_PATH environment variable).  The
serious problem is on a system or applications with an old hdf5 library
(e.g. 1.6).  Then you need an application such as h5repack built against a
recent hdf5 to repack the file with compressions that are known to the old
application, but somewhere you need a new hdf5 installation to make this
work.

  This will make a good test case to document in detail on the
hdrmx.medsbio.org website as a tutorial both for users and sysadmins.

  Regards,
    Herbert

On Fri, Jun 24, 2016 at 8:45 AM, Jemian, Pete R. <jemian at anl.gov> wrote:

> Eugen
>
> First off, I (representing the user who asked me about this) want to read
> that file with the HDF5 plugin in IgorPro on a Mac.
>
> Adding the compression library to this toolset is not trivial.  I must
> persuade the 3rd party developer of that plugin to make this addition.
>
> I also want to read that file with the h5py package in python on mac,
> Windows, and Linux.
>
> We are on the leading edge here but our users have already begun to
> struggle with reading these files.  Until the compression filters are part
> of common installations, a converter tool is needed.
>
> Pete
>
>
>
>
> Sent from a device with a dreadfully small screen
>
>
> -------- Original message --------
> From: Eugen Wintersberger <eugen.wintersberger at desy.de>
> Date: 6/24/2016 3:05 AM (GMT-06:00)
> To: "Jemian, Pete R." <jemian at anl.gov>, "Herbert J. Bernstein" <
> yayahjb at gmail.com>, Discussion forum for the NeXus data format <
> nexus at nexusformat.org>
> Cc: Tobias.Richter at diamond.ac.uk, jjakoncic at bnl.gov,
> andreas.foerster at dectris.com, rsweet at bnl.gov
> Subject: Re: [Nexus] how to read files from Eiger?
>
> Hi Pete,
>
> On which Linux systems do you want to read the files. For Ubuntu 14.04 and
> Debian 8  we have packages available.
> On 06/23/16 19:27, Pete Jemian wrote:
>
> Very helpful.  Thanks for the quick turnaround.
> Now I go off in a corner with an internet connection for quiet time ...
>
> On 6/23/2016 12:25 PM, Herbert J. Bernstein wrote:
>
> Dear Pete,
>
>   This is a solvable problem, but requires some preliminary software
> installations.  First, be sure your HDF5 kit is reasonably recent and has
> filter plugin support.  Then pickup and install both the LZ4 and BitShuffle
> filters.  Now make sure HDF5_PLUGIN_PATH is defined to point to the
> directory containing the filter libraries.  Then h5dump should be able to
> handle the Eiger files.
>
> For the current files BittShuffle is not required. Most probably required
> for latter releases of the detector?
>
> regards
>    Eugen
>
>
>   Eugen is gathering the filters on github.  I am gathering other software
> and links to test data and templates at HDRMX.medsbio.org.  There is a
> link for the hdrmx-bb list on the page.   Please post comments, questions
> and suggestions there.
>
>   Regards,
>     Herbert
>
> Sent from my iPad
>
> On Jun 23, 2016, at 12:48, Pete Jemian <prjemian at gmail.com>
> <prjemian at gmail.com> wrote:
>
>
> We have received a linked fileset from an Eiger 1M Anderson  want to read
> it.
> Cannot seem to progress past knowing that there is data.
> Here is an excerpt from h5dump (version 1.8.16):
>
>        DATASET "data" {
>           DATATYPE  H5T_STD_U32LE
>           DATASPACE  SIMPLE { ( 1, 1065, 1030 ) / ( H5S_UNLIMITED, 1065,
> 1030 ) }
>           DATA {h5dump error: unable to print data
>
>           }
>           ATTRIBUTE "image_nr_high" {
>              DATATYPE  H5T_STD_I32LE
>              DATASPACE  SCALAR
>              DATA {
>              (0): 1
>              }
>           }
>           ATTRIBUTE "image_nr_low" {
>              DATATYPE  H5T_STD_I32LE
>              DATASPACE  SCALAR
>              DATA {
>              (0): 1
>              }
>           }
>        }
>
> It's the error: "h5dump error: unable to print data"
>
> Is this the case where we have the wrong version of the HDF5 libraries?
> Is there a tool to render such files compatible with the 1.8 libraries?
> If so, where is that tool?
>
> Pete
>
> _______________________________________________
> NeXus mailing list
> NeXus at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus
>
>
>
> --
> Dr. Eugen Wintersberger
>
> FS-EC
> DESY
> Notkestr. 85
> D-22607 Hamburg
> Germany
>
> E-Mail: eugen.wintersberger at desy.de
> Telefon: +49-40-8998-1917
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nexusformat.org/pipermail/nexus/attachments/20160624/549448b0/attachment.html>