Skip to main content
eScholarship
Open Access Publications from the University of California

An In-Depth I/O Pattern Analysis in HPC Systems

Published Web Location

https://sdm.lbl.gov/oapapers/hipc2021-bang.pdf
No data is associated with this publication.
Creative Commons 'BY' version 4.0 license
Abstract

High-performance computing (HPC) systems consist of thousands of compute nodes, storage systems and high-speed networks, providing multiple layers of I/O stack with high complexity. By adjusting the diverse configuration settings that HPC systems provide, the I/O performance of applications can be improved. However, it is challenging to identify the optimal configuration settings without a thorough knowledge of the system, as each of the different I/O characteristics of applications can be an important factor for parameter decision. In this paper, we use multiple machine learning approaches to perform an in-depth analysis on I/O behaviors of HPC applications and to search for the optimal configuration settings for jobs sharing similar I/O characteristics. Improved by maximum 0.07 R-squared score, our results in overall show that jobs run on the HPC systems can obtain the predicted I/O performance for different configuration parameters with a high accuracy, using the proposed machine learning-based prediction models.

Item not freely available? Link broken?
Report a problem accessing this item