UC Davis

The quest for artificial general intelligence is fundamentally a quest for robust machine reasoning—the ability of a model to transcend simple pattern matching and perform multi-step logical inference. This dissertation argues that the bottleneck of current machine reasoning lies not just in model scale, but in the nature of the underlying representations the models utilize. We explore the evolution of these representations from static linguistic indices to dynamic, latent multimodal spaces, and finally to learned internal attention policies. The first part of this work addresses the limitations of categorical representations in Natural Language Processing. We introduce LITE, a framework that reformulates entity typing as a Natural Language Inference (NLI) problem. By mapping diverse entity types into a semantic hypothesis space rather than fixed indices, we demonstrate that models can achieve superior generalization to unseen concepts, establishing that reasoning is more effective when grounded in rich semantic representations. In the second part, we diagnose the fragility of these reasoning chains. Through the EUREQA benchmark, we uncover a systemic vulnerability in Large Language Models (LLMs): the tendency to follow "deceptive semantic shortcuts." We prove that models often rely on superficial entity associations rather than faithful multi-hop logic. This finding motivates the need for reasoning mechanisms that are more deeply grounded in the input data rather than biased linguistic priors. The third part of the dissertation extends these insights into the multimodal domain. We first propose Semantic-Clipping (SEMCLIP), a framework that uses textual semantics to dynamically select relevant visual representations, enhancing efficiency and fine-grained reasoning without retraining. We then advance this concept with Latent Visual Reasoning (LVR), a novel paradigm where reasoning occurs directly within the visual embedding space. By training Multimodal LLMs to generate latent states that reconstruct key visual features, we move beyond surface-level text rationales, enabling models to "think" in visual tokens. Finally, we propose Reinforced Attention Learning (RAL) to optimize the fundamental mechanics of these representations. By applying policy-gradient reinforcement learning directly to internal attention distributions, we shift the optimization objective from what the model generates to where it attends. This approach, coupled with On-Policy Attention Distillation, provides a principled method for cross-modal alignment that outperforms standard token-based distillation. Collectively, the contributions of this dissertation demonstrate that by advancing how models represent, select, and attend to information, we can move toward machine reasoning systems that are more efficient, less prone to hallucination, and capable of sophisticated latent inference across modalities.