Maintenance of both the HDF4 and HDF5 libraries and utilities are at the core of NCSA’s mission to support EOS activities, and includes the following tasks.
Subtask
|
Status
| 3.2.1Add features and correct errors
Errors and feature requests will be prioritized in consultation with ESDIS, ECS, and users, and addressed in a timely manner. The addition of features requires changes in interfaces, and this means keeping the C, Fortran, Java and C++ APIs up to date. It requires keeping the documentation, test suites and configurations current.
|
The list of known bugs in HDF4 was prioritized according to their effect on EOSDIS users. All high priority bugs including better support for SZIP compression were fixed for HDF4.2r1 which was released in February, 2005.
Similarly, all major HDF5 bugs and features were prioritized and fixed in the 1.6.4 and 1.6.5 releases (March and November, 2005, respectively), with priority given to those that most affect the HDF-EOS community.
New features and multiple bug fixes were included in the HDF Java products version 2.3 released in November 2005
Details of these releases can be found in HDF Newsletters 82, 83, 84, 85 and 86 at http://hdf.ncsa.uiuc.edu/newsletters/.
During 2005 the HDF5 team has been working towards the 1.8.0 HDF5 release.
Along with many new features HDF5 1.8.0 will include a new way of error handling (http://hdf.ncsa.uiuc.edu/workshops/HDF-EOS9/Presentations/Wed/HDF5%201.8%20Presentations/) and new n-bit and scale+offset compression methods.
Applications like HDF-EOS5 will benefit from improved error handling in HDF5. This mechanism provides a standard way of error handling and facilitates integration of application specific errors with the HDF5 error stack; the HDF team made sure to make new Error APIs forward compatible with the current release of the HDF-EOS5 library.
n-bit and scale+offset compression methods have been requested by many HDF-EOS and especially NPOESS application developers. Performance studies were presented at the HDF-EOS IX Workshop (http://hdf.ncsa.uiuc.edu/workshops/HDF-EOS9/Presentations/Fri/HDF5-nbit-scaleoffset.ppt ,
http://hdf.ncsa.uiuc.edu/workshops/HDF-EOS9/Presentations/Fri/scale-offset%20performance-posterpdf.pdf)
|
3.2.2Szip compression
The Szip compression modules are a special case, as they involve close collaboration with the University of Idaho, as well as special licensing constraints on the Szip encoder. The HDF Group will make certain that Szip is fully supported for those who cannot use the encoder, as well as for those who can
|
During 2005, the HDF Helpdesk received several SZIP related questions, bugs reports that helped to improve the HDF/SZIP documentation and implementation, especially in HDF4. HDF4/SZIP improvements will be introduced in the HDF4.2r2 release.
Information about SZIP support in the HDF libraries can be found at http://hdf.ncsa.uiuc.edu/doc_resource/SZIP/. SZIP/HDF performance studies are available from http://hdf.ncsa.uiuc.edu/RFC/SZIP/SZIPperformanceStudy.pdf
SZIP compression implemented in the HDF libraries and tools has become very popular among users inside and outside the HDF-EOS community.
|
Software will be maintained on, or ported to, all systems of importance to EOS. This also involves upgrading configurations and testing regimes. It is anticipated that the next six years will see increasing use of high performance systems such as Linux clusters.
|
During 2005 we added SZIP, HDF4 and HDF5 support for the following platforms at users’ requests:
-
Solaris 2.9
-
Solaris 2.10
-
HP-UX 11.23
-
Cray X1
-
CYGWIN
Support for the following platforms was dropped:
Improvements were done to the HDF5 Fortran library configuration that allow automatic discovery and mapping of Fortran data types to C data types, and to eliminate the need for a manual port for each new Fortran compiler or OS system. The same mechanism will be implemented in the HDF4.2r2 release.
During 2005 we also added support for g95 to the HDF5 development branch and latest versions of the PGI compilers.
The HDF team has been working on improvements for Windows maintenance and support. HDF products are tested daily on the Windows platforms with automatic batch scripts; Windows support documentation can be found at http://hdf.ncsa.uiuc.edu/windows.html.
| 3.2.4Documentation
The HDF group will prepare documentation in a timely manner, including User’s Guides for libraries and utilities, and an up-to-date reference manual at the time of each new release of the NCSA HDF library.
|
HDF documentation is available with every major release of the HDF software in the HTML and PDF formats. Work on the HDF5 User’s Guide is still in progress.
During 2005 documentation work focused on the HDF5 1.8.0 release. New features implemented in the 1.7 branch (error handling, metadata cache, n-bit and scale+offset filters, dimension sacles, etc) have been added to the HDF5 User’s Guide. See http://hdf.ncsa.uiuc.edu/HDF5/doc_dev_snapshot/H5_dev/ for the new chapters.
New C APIs were added to the HDF5 Reference Manual along with the HL, Dimension Scales and Packets APIs.
Multiple bugs were fixed in the HDF5 Reference Manual and User’s Guide for the HDF5 1.6.4 and 1.6.5 releases.
| 3.2.5Conduct periodic releases
Produce at least one new release for HDF4 to keep up with OS and language upgrades, bug fixes, new features, and new platforms.
Produce approximately three HDF5 releases.
Release other products as appropriate, such as the HDF5 high level libraries, and the Java libraries.
Conduct periodic releases
n 2002. aced the earlier
of a single data model that covers both formatstables.
|
HDF4.2r1 was released in February, 2005. Highlights include:
-
All Szip compression feature requirements were added and any remaining defects were fixed.
-
hdiff and hrepack were added for Windows.
-
Config, build, testing procedures were improved.
-
The h4fc utility was fixed.
-
New compilers and platforms included Mac OS X, Fortran IBM xlf v. 8.1, Absoft f95 v. 8.2, AMD Opteron, Cray TS IEEE, Linux 2.4, Absoft Fortran f95 v. 9.0, PGI C and Fortran, and Intel C and Fortran.
The HDF5 1.6.4 and 1.6.5 releases occurred in March and November, respectively. Highlights include:
-
High-Level (HL) library changes, including addition of some new C APIs, the building of the HL library by default.
-
Library now built and tested with SZIP 2.0.
-
Many changes to improve library performance, especially for variable length types and metadata cache.
-
H5jam – a new utility that allows a text file to be added to the "user block" at the beginning of an HDF5 file.
-
Systems added: Solaris 2.10, Cray X1, Cray XT3, HP 64-bit (HPUX 11.23)
-
Compilers added: gcc 4.*, g95
-
Platforms to be dropped in future releases include:
-
Operating systems: Solaris 2.8, HPUX B.11.00
-
Compilers: We use the latest versions of vendors compilers as they become available and drop the previous ones.
HDF Java Products version 2.3 were released in November 2005. It included bug fixes and a few new features in HDFView described in section 3.3.3 below.
| 3.2.6Quality assurance
NCSA will continue to make QA an important component of all activities. Areas that will receive special emphasis are the library testing operations, documentation, the software development process, and software development standards.
|
Considerable work continues to go into the HDF regression tests, which are run daily on a critical set of platforms, including platforms running SGI, Solaris, AIX, HP, and Linux, as well as Linux clusters and Windows platforms.
Both the sequential and parallel HDF5 libraries are tested with the latest stable versions of MPICH and vendor-provided MPI-IO libraries. Various C and Fortran compilers (Intel, PGI, Absoft) are tested rigorously along with the multiple versions of the GNU C compiler. Periodically the HDF libraries are tested on Windows, Crays, SGI Altix and Compaq cluster machines.
We also periodically test the HDF-EOS5 library with available HDF5 snapshots. HDF-EOS5 was tested with the 1.6.4 and 1.6.5 HDF5 releases to make sure that new features/bug fixes do not break current HDF-EOS releases.
The HDF bugs database is revised bi-weekly and new requests are prioritized and assigned to the developers. We keep improving our test coverage in the HDF product test suites including test coverage of different configuration options and compilation flags.
HDF group members continue to work closely with the users who reported bugs or requested new features and proactively seek feedback when a bug is fixed or a new feature is added.
For each release, the HDF library documentation is reviewed by the developers, and the QA and support group.
Starting with the HDF5 1.6.5 release, the HDF release process was modified to include an md5 file checksum and a PGP signature of the release to ensure that the distribution that is downloaded is actually the software that was released by NCSA.
|
Share with your friends: |
