Lawrence Berkeley National Laboratory

The vast amounts of data generated by large High Energy Physics (HEP) experiments pose significant challenges for data management and analysis. To mitigate these challenges, distributed caching systems, such as XCache, are used as regional in-network caches to buffer recently accessed data files. By reducing the need for repeated file transfers, in-network caching decreases data access latency and enhances analysis efficiency. To better understand the impact of data file popularity on cache effectiveness, we conducted a study of operational logs from Southern California from June 2020 - April 2025, comprising approximately 35 million file requests. Our extensive exploratory data analysis revealed that a small subset of datasets accounts for a disproportionate number of access requests, suggesting that prioritizing these popular datasets in the cache could simplify cache replacement policies and improve access efficiency. However, our analysis also showed that dataset popularity exhibits significant temporal variability, making it challenging to predict future access patterns. To address this, we developed a Long Short-Term Memory (LSTM) neural network model to forecast dataset access counts and volumes, training and testing this on the last year of data available (April 2024 - April 2025). Our model achieves a mean relative RMSE of 0.406 across the 20 most popular datasets, demonstrating its effectiveness in capturing general access trends. Although further validation is necessary, our results indicate the potential of LSTM-based predictions to enable the practical implementation of a “pinning” approach and improve data access efficiency for large-scale HEP analyses.