[Nexus-developers] Memory use during putslab and getslab

[Ray Osborn]

Hi Mark,

On Mar 17, 2011, at 3:31 AM, Mark Koennecke wrote:

> I am missing one important morsel of information: when creating the array, what do you specify as the chunksize?
> The recommendation is to set the chunksize to approximately  the size of the slabs you are going to read and write.
> If the whole dataset is just one chunk this would explain your observations.

Thanks for the tip. The chunk size is set automatically in tree.py to the last dimension, when it calls compmakedata. I was testing it with only one dimension, so that may be it. I will test it with multidimensional arrays, which is the intended use anyway.

I'll get back to you when I've done more tests.

>> Also, is there a way to make the undefined data get returned as NaNs? Presumably, HDF5 is returning a default value when it finds no stored data, so can that default value be changed, and could this be an option in the NeXus API?
>> 
>>  
> We have to look into this...

I think it would be very useful because I presume otherwise there is no way of knowing what the meaning of the zero is. Does HDF5 automatically fill a chunk with zeros? 

Ray
-- 
Dr Raymond Osborn
Group Leader, Neutron & X-ray Scattering
Materials Science Division
Argonne National Laboratory
Argonne, IL 60439, USA
Phone: +1 (630) 252-9011
Email: ROsborn at anl.gov