[Nexus] Slow writing in HDF4 and HDF5 with many NXdata blocks

["V. Armando Solé"]

On 16/04/2012 16:30, Mark Koennecke wrote:
> Hi Emmanuel,
>
> On 04/13/2012 03:41 PM, Emmanuel FARHI wrote:
>> Dear NeXus users,
>>
>> I'm creating HDF files with many NXdata blocks (about 850).
>
> I wonder if this a good approach. I would rather append the data to 
> one big array
> with an extendible dimension.

Even then one has to be very careful when using compression (his case).

The following is a *must read* for anybody using HDF5, compression 
and/or chunking::

http://www.hdfgroup.org/HDF5/doc/Advanced/Chunking/

http://www.hdfgroup.org/HDF5/doc/Advanced/Chunking/Chunking_Tutorial_EOS13_2009.pdf


For those wondering if they should read or not the above, just take a 
look at slide 37 of the last one.

Armando