[Nexus] Advice on using NeXus for neuroscience data?

[V. Armando Sole]

Hi Jeff,

If it is to be strictly followed, I find NeXus too rigid and cumbersome. 
However, some of the ideas are great.

Among the great points (at least in my opinion) you can find the concept 
of Application Defininition. I would recommend you the following:

1 - Figure out what information you absolutely need to describe your 
neurophysiology data. This is clearly the most critical point and you 
should think about it *without* considering HDF5 or any other format. 
Just figure out what you need. HDF5 is so versatile that it will 
certainly be up to the task.

2 - Only when you have that information, you can decide about if you 
want a flattened structure containing the information or a hierarchical 
one. Thanks to the link capability of HDF5, you can have a flattened 
structure with the information you need pointing to any other more 
structured information in the same file or even in separated files.

3 - Although you could manage by yourself, NeXus would be of help 
providing the tools to validate a file has properly implemented point 2 
(look for Application Definitions).

4 - If in addition to describe the data, you need to describe the 
associated instrumentation, then you could make use of other NeXus 
conventions.

Trust me, points 1 and 2 can keep you busy for quite a long time. Please 
do not make the mistake to think about the physical data format prior to 
have solved them.

I hope that helps.

Armando

On 01.10.2014 22:56, Jeff Teeters wrote:
> Hi,
> I'm helping to coordinate a project (called "Neurodata Without
> Borders", described below), which has the goal to develop a standard
> file format for neurophysiology data.   It seems likely that HDF5 will
> be used as the basis for the standard format, but the organization of
> data and metadata within the HDF5 file still must be determined.  I'm
> interested in any opinions or input about the following:
> a. Is NeXus potentially useful for domains unrelated to particle
> physics experiments?
> b. Are there any examples of NeXus being used in other domains?
> c. What other methods of specifying the organization of data in an
> HDF5 file might be applicable for this project?
> 
> Thanks,
> 
> Jeff Teeters
> Redwood Center for Theoretical Neuroscience
> UC Berkeley
> 
> ---------- Forwarded message ----------
> From: JEFF TEETERS <jteeters at berkeley.edu>
> Date: Mon, Aug 11, 2014 at 11:47 PM
> Subject: Seeking community input for Neurodata Without Borders
> To: Jeff Teeters <jteeters at berkeley.edu>
> Cc: Fritz Sommer <fsommer at berkeley.edu>
> 
> Dear Colleague,
> 
> The Neurodata Without Borders (NWB) project has just started. The
> project goal is to build a common data format for neurophysiology data
> from Allen Brain Institute, Janelia Farm and two labs from NYU and
> Caltech.  Although focusing on a limited set of use cases, the project
> also aims to develop products that will serve the broader community.
> 
> At this point we would like to solicit community input about
> ideas/approaches for designing a generalizable neurophysiology data
> format. If you are interested in contributing to this project, please
> review the project description at:
>  https://crcns.org/NWB [1]
> and fill out the questionnaire:
> https://docs.google.com/forms/d/1CNTd4M-8kQ_WhEZc7n7WxpTa0LOupt_q3z21E1fRxjM/viewform
> [2]
> 
> On the basis of the questionnaire responses and ensuing communication,
> we will organize the first hackathon meeting of the project, to be
> held November 20 – 22, 2014 (just after SfN) at Janelia Farm, in
> Ashburn, Virginia.  At this hackathon we will discuss in detail the
> requirements for a common format based on the project use cases and
> also discuss, compare and evaluate alternative techniques for
> implementing the common format.
> 
> More information about the project is available in a recent press
> release:
> http://www.kavlifoundation.org/kavli-news/prominent-us-research-institutions-announce-collaboration-toward-sharing-and
> [3]
> 
> Please forward this email to anyone you know with relevant expertise
> who may be interested in contributing to this project.
> 
> Thank you,
> Fritz Sommer and Jeff Teeters
> Redwood Center for Theoretical Neuroscience
>  UC Berkeley
> 
> 
> Links:
> ------
> [1] https://crcns.org/NWB
> [2]
> https://docs.google.com/forms/d/1CNTd4M-8kQ_WhEZc7n7WxpTa0LOupt_q3z21E1fRxjM/viewform
> [3]
> http://www.kavlifoundation.org/kavli-news/prominent-us-research-institutions-announce-collaboration-toward-sharing-and
> 
> _______________________________________________
> NeXus mailing list
> NeXus at nexusformat.org
> http://lists.nexusformat.org/mailman/listinfo/nexus