From Persisting Bytes to Retrieving Context: Crash-Consistent PMEM Systems and Efficient Context Retrieval for LLMs
- Soh, Yun Joon
- Advisor(s): Zhao, Jishen
Abstract
Information is most valuable when it can be reliably persisted and efficiently retrieved. This thesis explores two complementary challenges in information management: ensuring crash consistency for modern storage systems such as persistent memory and enabling accurate and efficient long-context retrieval for large language models (LLMs).
To address the first challenge, the thesis presents ENTS, a programming library that eliminates manual cacheline flush and memory barrier operations, and SSAPP, a compiler-assisted framework that automates crash-consistency enforcement. These systems improve the performance and usability of persistent memory, achieving up to 2.1x and 1.8x speedups, respectively.
To address the second challenge, the thesis introduces YOURA, a training-free, attention-based retrieval technique that leverages off-the-shelf LLMs for long-context question answering. YOURA improves the accuracy and inference throughput by up to 15% and 30%, respectively. While it incurs additional preprocessing overhead, this cost can be mitigated through a co-designed strategy that persists the preprocessed context such as key-value caches on storage for efficient reuse across multiple queries.
These contributions demonstrate how co-designed systems can advance the state of the art in both physical and semantic data-flow for the modern applications.