UC Irvine

Large language models have advanced rapidly in recent years, narrowing the gap between human and machine performance on many established benchmarks and, in some cases, exceeding human performance. As their capabilities have grown, these systems have become increasingly embedded in both professional and everyday life. They are now used to support work across domains such as education, healthcare, public policy, technology, and professional services, while also being used in more personal contexts, including companionship and decisions about everyday life. However, the very capabilities that make these models useful also make their limitations harder to evaluate, as failures can be subtle, context-dependent, and difficult to predict. They may excel at highly difficult tasks while failing in seemingly simple settings, especially when faced with new or unexpected contexts. The growing number of publicly reported AI incidents, including cases where AI systems have caused or nearly caused real-world harm, underscores the need to better understand how these systems fail and build more robust models. In this thesis, we argue that understanding and improving large language models requires targeted evaluation in novel settings. Focusing on emerging language use, new model architectures, and reasoning paradigms, we develop evaluations that expose specific limitations of LLMs and use those insights to guide more robust model development. In the first part of this thesis, we study emerging and socially consequential language use through the problem of misgendering non-binary individuals. We introduce Misgendered, a framework for evaluating language models’ ability to correctly use gender-neutral pronouns and neopronouns, and MisgenderMender, a community-informed task and dataset for detecting and correcting text-based misgendering. In the second part, we examine how emerging model architectures create new questions for evaluation and require methods tailored to their underlying mechanisms. Focusing on Mamba state space models, whose fixed-size hidden states require compressing context rather than storing each token individually as in transformers, we train an autoencoder to directly probe information retention in the hidden state and show that retention is uneven, with particularly poor preservation of numeric information, motivating architecture-specific recommendations for improving information retention. In the final part of the thesis, we target mathematical reasoning through the lens of consistency. Although chain-of-thought reasoning has substantially improved the performance of large language models, generated solutions can still contain contradictions and unsupported intermediate claims. We show that enforcing formal consistency constraints helps diagnose these failures and improve model-generated mathematical reasoning. Together, these contributions demonstrate that emerging AI paradigms require targeted evaluations that can reveal specific model failures, and guide the development of more reliable systems.