UCLA

Reasoning is a fundamental capability of human intelligence that enables complex problem-solving and decision-making. Recent advances in Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse tasks, yet their reasoning abilities in complex, long-horizon scenarios remain inadequately understood. This thesis systematically investigates and enhances the reasoning capabilities of LLMs across three progressively complex domains: constrained game playing, task and motion planning, and sequential manipulation in partially known environments.First, we explore LLM reasoning in the structured domain of chess, where we demonstrate that language explanations combining long-term strategic thinking and short-term tactical analysis significantly enhance model performance. Through the MATE dataset of 1 million annotated chess positions, we show that fine-tuned models incorporating strategy and tactic annotations outperform state-of-the-art commercial LLMs by 24.2%, establishing that explicit reasoning guidance improves decision-making quality.Second, we present LLM3 , a framework that leverages LLMs as domain-independent interfaces between symbolic task planning and continuous motion planning. By categorizing motion planning feedback into collision and unreachability failure modes, LLM3 enables iterative refinement of action sequences and parameters. Experimental results demonstrate a 91.3% success rate across realistic scenarios, with 36.1% reduction in travel distance compared to baselines, highlighting the effectiveness of LLM-based failure reasoning.Third, we introduce EPoG, which integrates exploration and sequential manipulation planning on graph-based scene representations for partially known environments. By employing LLMs for both informed exploration prioritization and situated replanning, EPoG naturally combines information gathering with task execution. Across 46 household scenes and 5 long-horizon tasks, EPoG achieves superior performance while reducing exploration overhead by 40% and travel distance by 36.2%.Collectively, this thesis makes three primary contributions: (1) demonstrating that language-based explanations enhance LLM reasoning across domains of increasing complexity, (2) establishing effective methods for integrating LLM reasoning with classical planning algorithms through structured feedback mechanisms, and (3) showing that LLMs can provide domain-independent heuristics that improve both planning efficiency and adaptability to environmental uncertainty. These findings advance our understanding of LLM reasoning capabilities and provide practical frameworks for deploying LLMs in complex embodied AI applications.