UC Berkeley

This dissertation focuses on developing systematic frameworks to understand and improve the reasoning capabilities of large language models (LLMs) in two axes: reliability and efficiency. In terms of reliability, we study how LLMs learn parametric knowledge during training, and how they combine separate atomic knowledge to deduce new conclusions during test time. We develop theoretical frameworks and use out-of-context reasoning as a concrete lens to analyze model behavior, providing both theoretical explanations and empirical evidence that LLMs can hallucinate or fail to generalize systematically. In terms of efficiency, we develop novel paradigms for test-time scaling to improve LLMs' capabilities and reduce inference cost. We focus on latent space reasoning, particularly chain-of-continuous-thought, demonstrating its theoretical advantages where continuous thoughts can maintain a superposition of multiple solutions and thus enable implicit parallel thinking. We also study how this mechanism emerges during training without explicit supervision. Together, the results in this thesis provide theoretical foundations for understanding and improving LLM reasoning, highlighting the importance of models' reliability and efficiency, especially when applying those models to important fields and challenging tasks.