[Nexus] NAPI future - input from the McStas team

Fri Oct 23 20:03:23 BST 2020

Dear Mark, Freddie and everyone else.

McStas (and thus McXtrace) has had partial NeXus support since ~ year 2000 and full support since ~2008. McStas is written in (and generates) C, and hence it was easy and convenient for us to make use of the NeXus C-API trhough napi.h.

While we do have a long term plan to transition to writing through lower-level HDF calls, it is unclear when we will have resources and time for such a development effort. The napi is btw, the basis of our hooks to Mantid.

Best,

Peter W et. al.

Here are the formal answers to the survey:
____________

Abandoning the support for NAPI for bugfix or compatibility with future linux/library/compiler versions will cause me problems at the following level:

[x] Critical: I am using NAPI in critical data acquisition or data reduction/analysis software
[x] Major: important applications may not be usable at some future date
[] Minor: inconvenience, I have alternatives/can live with this
[] None: either i am not using NAPI, or I have copies of all the dependencies that I build/maintain myself

[] I want the NIAC to continue developing NAPI and make it support more HDF5 features. Please list the
 HDF5 feature you miss in NAPI.

Which of the NAPI file format bindings do you use:

[] HDF4
[x] HDF5
[] XML

Which of the NAPI language bindings do you use?

[x] ANSI-C
[] Fortran
[] C++
[] Java
[] IDL
[] Swig generated bindings to Tcl, perl, ...

Which of the NAPI utilities do you use?

[x] NXbrowse
[] NXingest
[] NXtranslate
[] NXconvert
[] NXdump
[] NXsummary

General

[x] I am willing to help maintain NAPI

On 22 Oct 2020, at 17.08, nexus-request at shadow.nd.rl.ac.uk<mailto:nexus-request at shadow.nd.rl.ac.uk> wrote:

Date: Thu, 22 Oct 2020 14:08:40 +0000
From: "Koennecke Mark (PSI)" <mark.koennecke at psi.ch<mailto:mark.koennecke at psi.ch>>
To: Discussion forum for the NeXus data format <nexus at nexusformat.org<mailto:nexus at nexusformat.org>>
Subject: [Nexus] NAPI future
Message-ID: <E847FB16-12D2-4C5C-A6F7-7950CD64F0D4 at psi.ch<mailto:E847FB16-12D2-4C5C-A6F7-7950CD64F0D4 at psi.ch>>
Content-Type: text/plain; charset="us-ascii"

Dear NeXus-API users,

The NeXus International Advisory Committee (NIAC) is contemplating dropping support for the NeXus-API (NAPI, https://manual.nexusformat.org/napi.html?highlight=napi ),
his is a library for accessing NeXus data files. Before we make a decision, we would like to determine how many people are still using it. If you still rely on this API,
please let the NIAC know! In order to focus our efforts better, we would also like to have some information about which parts of NAPI are still used.

But first some history: The NeXus-API was conceived shortly after the inception of NeXus as a means to simplify the access to files in the then prevalent HDF4 format. The HDF4 API is very complex and NAPI abstracted a lot of that complexity away. At that time the NAPI was a major selling point for NeXus.

Years later the HDF group presented HDF-5 and it was felt that the NAPI should support HDF-5 too. And this being a transition period, it should support both HDF4 and HDF5 transparently. Consequently the NAPI was refactored to allow other file formats and HDF5 was integrated.

Some more years later, the community wished to have a human readable ASCII version of the NeXus file format. XML was in favour at the time and thus NAPI was extended to support an XML version of NeXus using the mxml library.

Moving to the current day, the simpler HDF5 native interface and python packages such as h5py means many people are writing NeXus format files directly with these interfaces rather than using NAPI. This lead to the decision by the NIAC to place the NAPI in maintenance only mode.

The NAPI now requires some attention, most notably to make it work with newer versions of the underlying libraries such as mxml-3.x which introduced a breaking API change and is now present on many recent linux distributions. We are preparing a new NAPI release to address this. But the NIAC as an organization is relying on volunteers to get its work done and is hard pressed to maintain NAPI, so we wish to determine where best to divert our efforts.

To help the NIAC make the most informed decisions, please could you answer a few questions. Please place an X between the square brackets in order to mark statements which apply.

## Questions

Abandoning the support for NAPI for bugfix or compatibility with future linux/library/compiler versions will cause me problems at the following level:

[] Critical: I am using NAPI in critical data acquisition or data reduction/analysis software
[] Major: important applications may not be usable at some future date
[] Minor: inconvenience, I have alternatives/can live with this
[] None: either i am not using NAPI, or I have copies of all the dependencies that I build/maintain myself

[] I want the NIAC to continue developing NAPI and make it support more HDF5 features. Please list the
 HDF5 feature you miss in NAPI.

Which of the NAPI file format bindings do you use:

[] HDF4
[] HDF5
[] XML

Which of the NAPI language bindings do you use?

[] ANSI-C
[] Fortran
[] C++
[] Java
[] IDL
[] Swig generated bindings to Tcl, perl, ...

Which of the NAPI utilities do you use?

[] NXbrowse
[] NXingest
[] NXtranslate
[] NXconvert
[] NXdump
[] NXsummary

General

[] I am willing to help maintain NAPI

The NeXus validator: cnxvalidate, is missing in this list. This is a separate product to NAPI and the NIAC is committed to maintaining it. It uses the HDF5 API directly rather than NAPI.

Thank you for your time filling in this questionnaire!

Please send replies to  nexus-committee at nexusformat.org<mailto:nexus-committee at nexusformat.org>

If you have further concerns or comments regarding the NAPI, please inform your NIAC representative - these are listed at  https://www.nexusformat.org/NIAC.html

If your facility is using NeXus, but has no representation on the NIAC, please consider joining the NIAC.

Best Regards,

  Freddie Akeroyd
  Mark Koennecke
  on behalf of the NIAC

------------------------------

Message: 2
Date: Thu, 22 Oct 2020 14:08:53 +0000
From: "Koennecke Mark (PSI)" <mark.koennecke at psi.ch<mailto:mark.koennecke at psi.ch>>
To: Discussion forum for the NeXus data format <nexus at nexusformat.org<mailto:nexus at nexusformat.org>>
Subject: [Nexus] NAPI future
Message-ID: <A9C14699-06DA-4312-B39D-84A0A5C714E1 at psi.ch<mailto:A9C14699-06DA-4312-B39D-84A0A5C714E1 at psi.ch>>
Content-Type: text/plain; charset="us-ascii"

Dear NeXus-API users,

The NeXus International Advisory Committee (NIAC) is contemplating dropping support for the NeXus-API (NAPI, https://manual.nexusformat.org/napi.html?highlight=napi ),
his is a library for accessing NeXus data files. Before we make a decision, we would like to determine how many people are still using it. If you still rely on this API,
please let the NIAC know! In order to focus our efforts better, we would also like to have some information about which parts of NAPI are still used.

But first some history: The NeXus-API was conceived shortly after the inception of NeXus as a means to simplify the access to files in the then prevalent HDF4 format. The HDF4 API is very complex and NAPI abstracted a lot of that complexity away. At that time the NAPI was a major selling point for NeXus.

Years later the HDF group presented HDF-5 and it was felt that the NAPI should support HDF-5 too. And this being a transition period, it should support both HDF4 and HDF5 transparently. Consequently the NAPI was refactored to allow other file formats and HDF5 was integrated.

Some more years later, the community wished to have a human readable ASCII version of the NeXus file format. XML was in favour at the time and thus NAPI was extended to support an XML version of NeXus using the mxml library.

Moving to the current day, the simpler HDF5 native interface and python packages such as h5py means many people are writing NeXus format files directly with these interfaces rather than using NAPI. This lead to the decision by the NIAC to place the NAPI in maintenance only mode.

The NAPI now requires some attention, most notably to make it work with newer versions of the underlying libraries such as mxml-3.x which introduced a breaking API change and is now present on many recent linux distributions. We are preparing a new NAPI release to address this. But the NIAC as an organization is relying on volunteers to get its work done and is hard pressed to maintain NAPI, so we wish to determine where best to divert our efforts.

To help the NIAC make the most informed decisions, please could you answer a few questions. Please place an X between the square brackets in order to mark statements which apply.

## Questions

Abandoning the support for NAPI for bugfix or compatibility with future linux/library/compiler versions will cause me problems at the following level:

[] Critical: I am using NAPI in critical data acquisition or data reduction/analysis software
[] Major: important applications may not be usable at some future date
[] Minor: inconvenience, I have alternatives/can live with this
[] None: either i am not using NAPI, or I have copies of all the dependencies that I build/maintain myself

[] I want the NIAC to continue developing NAPI and make it support more HDF5 features. Please list the
HDF5 feature you miss in NAPI.

Which of the NAPI file format bindings do you use:

[] HDF4
[] HDF5
[] XML

Which of the NAPI language bindings do you use?

[] ANSI-C
[] Fortran
[] C++
[] Java
[] IDL
[] Swig generated bindings to Tcl, perl, ...

Which of the NAPI utilities do you use?

[] NXbrowse
[] NXingest
[] NXtranslate
[] NXconvert
[] NXdump
[] NXsummary

General

[] I am willing to help maintain NAPI

The NeXus validator: cnxvalidate, is missing in this list. This is a separate product to NAPI and the NIAC is committed to maintaining it. It uses the HDF5 API directly rather than NAPI.

Thank you for your time filling in this questionnaire!

Please send replies to  nexus-committee at nexusformat.org<mailto:nexus-committee at nexusformat.org>

If you have further concerns or comments regarding the NAPI, please inform your NIAC representative - these are listed at  https://www.nexusformat.org/NIAC.html

If your facility is using NeXus, but has no representation on the NIAC, please consider joining the NIAC.

Best Regards,

 Freddie Akeroyd
 Mark Koennecke
 on behalf of the NIAC

------------------------------

Message: 3
Date: Thu, 22 Oct 2020 14:49:42 +0000
From: "Chang, Peter (DLSLtd,RAL,LSCI)" <Peter.Chang at Diamond.ac.uk<mailto:Peter.Chang at Diamond.ac.uk>>
To: "'Discussion forum for the NeXus data format'"
<nexus at shadow.nd.rl.ac.uk<mailto:nexus at shadow.nd.rl.ac.uk>>
Subject: Re: [Nexus] NAPI future
Message-ID:
<DB6PR0102MB26804B9DFE5BDAC794244B12B91D0 at DB6PR0102MB2680.eurprd01.prod.exchangelabs.com<mailto:DB6PR0102MB26804B9DFE5BDAC794244B12B91D0 at DB6PR0102MB2680.eurprd01.prod.exchangelabs.com>>

Content-Type: text/plain; charset="us-ascii"

Dear Mark and others,

Diamond Light Source does not use the NAPI.

Best regards,
Peter

-----Original Message-----
[...]
## Questions

Abandoning the support for NAPI for bugfix or compatibility with future linux/library/compiler versions will cause me problems at the following level:

[] Critical: I am using NAPI in critical data acquisition or data reduction/analysis software
[] Major: important applications may not be usable at some future date
[] Minor: inconvenience, I have alternatives/can live with this
[x] None: either I am not using NAPI, or I have copies of all the dependencies that I build/maintain myself

[] I want the NIAC to continue developing NAPI and make it support more HDF5 features. Please list the
HDF5 feature you miss in NAPI.

Which of the NAPI file format bindings do you use:

[] HDF4
[] HDF5
[] XML

Which of the NAPI language bindings do you use?

[] ANSI-C
[] Fortran
[] C++
[] Java
[] IDL
[] Swig generated bindings to Tcl, perl, ...

Which of the NAPI utilities do you use?

[] NXbrowse
[] NXingest
[] NXtranslate
[] NXconvert
[] NXdump
[] NXsummary

General

[] I am willing to help maintain NAPI

The NeXus validator: cnxvalidate, is missing in this list. This is a separate product to NAPI and the NIAC is committed to maintaining it. It uses the HDF5 API directly rather than NAPI.

Thank you for your time filling in this questionnaire!

Please send replies to  nexus-committee at nexusformat.org<mailto:nexus-committee at nexusformat.org>

If you have further concerns or comments regarding the NAPI, please inform your NIAC representative - these are listed at  https://www.nexusformat.org/NIAC.html

If your facility is using NeXus, but has no representation on the NIAC, please consider joining the NIAC.

Best Regards,

 Freddie Akeroyd
 Mark Koennecke
 on behalf of the NIAC

_______________________________________________
NeXus mailing list
NeXus at nexusformat.org<mailto:NeXus at nexusformat.org>
https://lists.nexusformat.org/mailman/listinfo/nexus

------------------------------

Message: 4
Date: Thu, 22 Oct 2020 17:08:28 +0200
From: V. Armando Sol? <sole at esrf.fr>
To: nexus at nexusformat.org
Subject: Re: [Nexus] NAPI future
Message-ID: <10803012-d012-0fc2-c351-59e4f03c3740 at esrf.fr>
Content-Type: text/plain; charset=utf-8

Dear all,

The ESRF does not use the NAPI.

Best regards,

Armando

On 22/10/2020 16:08, Koennecke Mark (PSI) via NeXus wrote:
Dear NeXus-API users,

The NeXus International Advisory Committee (NIAC) is contemplating dropping support for the NeXus-API (NAPI, https://manual.nexusformat.org/napi.html?highlight=napi ),
his is a library for accessing NeXus data files. Before we make a decision, we would like to determine how many people are still using it. If you still rely on this API,
please let the NIAC know! In order to focus our efforts better, we would also like to have some information about which parts of NAPI are still used.

But first some history: The NeXus-API was conceived shortly after the inception of NeXus as a means to simplify the access to files in the then prevalent HDF4 format. The HDF4 API is very complex and NAPI abstracted a lot of that complexity away. At that time the NAPI was a major selling point for NeXus.

Years later the HDF group presented HDF-5 and it was felt that the NAPI should support HDF-5 too. And this being a transition period, it should support both HDF4 and HDF5 transparently. Consequently the NAPI was refactored to allow other file formats and HDF5 was integrated.

Some more years later, the community wished to have a human readable ASCII version of the NeXus file format. XML was in favour at the time and thus NAPI was extended to support an XML version of NeXus using the mxml library.

Moving to the current day, the simpler HDF5 native interface and python packages such as h5py means many people are writing NeXus format files directly with these interfaces rather than using NAPI. This lead to the decision by the NIAC to place the NAPI in maintenance only mode.

The NAPI now requires some attention, most notably to make it work with newer versions of the underlying libraries such as mxml-3.x which introduced a breaking API change and is now present on many recent linux distributions. We are preparing a new NAPI release to address this. But the NIAC as an organization is relying on volunteers to get its work done and is hard pressed to maintain NAPI, so we wish to determine where best to divert our efforts.

To help the NIAC make the most informed decisions, please could you answer a few questions. Please place an X between the square brackets in order to mark statements which apply.

## Questions

Abandoning the support for NAPI for bugfix or compatibility with future linux/library/compiler versions will cause me problems at the following level:

[] Critical: I am using NAPI in critical data acquisition or data reduction/analysis software
[] Major: important applications may not be usable at some future date
[] Minor: inconvenience, I have alternatives/can live with this
[X] None: either i am not using NAPI, or I have copies of all the dependencies that I build/maintain myself

[] I want the NIAC to continue developing NAPI and make it support more HDF5 features. Please list the
HDF5 feature you miss in NAPI.

Which of the NAPI file format bindings do you use:

[] HDF4
[] HDF5
[] XML

Which of the NAPI language bindings do you use?

[] ANSI-C
[] Fortran
[] C++
[] Java
[] IDL
[] Swig generated bindings to Tcl, perl, ...

Which of the NAPI utilities do you use?

[] NXbrowse
[] NXingest
[] NXtranslate
[] NXconvert
[] NXdump
[] NXsummary

General

[] I am willing to help maintain NAPI

The NeXus validator: cnxvalidate, is missing in this list. This is a separate product to NAPI and the NIAC is committed to maintaining it. It uses the HDF5 API directly rather than NAPI.

Thank you for your time filling in this questionnaire!

Please send replies to  nexus-committee at nexusformat.org

If you have further concerns or comments regarding the NAPI, please inform your NIAC representative - these are listed at  https://www.nexusformat.org/NIAC.html

If your facility is using NeXus, but has no representation on the NIAC, please consider joining the NIAC.

Best Regards,

 Freddie Akeroyd
 Mark Koennecke
 on behalf of the NIAC

_______________________________________________
NeXus mailing list
NeXus at nexusformat.org
https://lists.nexusformat.org/mailman/listinfo/nexus

------------------------------

Subject: Digest Footer

_______________________________________________
NeXus mailing list
NeXus at nexusformat.org
https://lists.nexusformat.org/mailman/listinfo/nexus

------------------------------

End of NeXus Digest, Vol 143, Issue 3
*************************************

Peter Kjær Willendrup
Forskningsingeniør, Specialkonsulent
Næstformand for DTU Fysik LSU

DTU Physics

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.nexusformat.org/pipermail/nexus/attachments/20201023/806a4940/attachment-0003.htm>