Lawrence Berkeley National Laboratory
H5Part: A Portable High Performance Parallel Data Interface for Particle Simulations
- Author(s): Adelmann, Andreas
- Ryne, Robert D.
- Siegerist, Cristina
- Shalf, John M.
- et al.
The very largest parallel particle simulations, for problems involving six dimensional phase space, generate vast quantities of data. It is desirable to store such enormous datasets efficiently and also to share data effortlessly between data analysis tools such as PartView~\cite FPAT082 and extensions to AVS/Express among other groups who are working on particle-based accelerator simulations. We define a very simple file schema built on top of HDF5~\cite hdf5hp (Hierarchical Data Format version 5) as well as an API that simplifies the reading/writing of the data to the HDF5 file format. HDF5 offers a self-describing machine-independent binary file format that supports scalable parallel I/O performance for MPI codes on computer systems ranging from laptops to supercomputers. The sample H5PartAPI is available for C, C++, and Fortran codes. The common file format will enable groups that using completely different simulation implementations to transparently share datasets and custom data analysis tools like Part View. We will show examples and benchmark data for various platforms.