UC San Diego

This dissertation explores the intersection of modeling and optimization in robotics, focusing on the development of efficient and effective systems for robotic manipulation. The primary objective is to study how to integrate modeling techniques with optimization processes, a concept we term "modeling-based optimization."

We first introduce a differentiable physics simulator for soft-body manipulation, demonstrating the power of environment modeling in policy learning. By simulating elastoplastic materials such as plasticine, we benchmark reinforcement learning (RL) and gradient-based optimization methods, highlighting the strengths and limitations of each approach. The findings reveal that while gradient-based methods excel in environments with well-modeled physics, they struggle with long-term planning and multi-stage tasks.

To address these challenges, we propose a reparameterized policy gradient method, which leverages latent variable models to facilitate exploration and avoid local minima. This approach integrates generative models to enhance policy expressiveness and improve performance in hard-exploration tasks. We further extend the concept of hierarchical policy modeling by introducing graph-based and vision-language-driven methods. These techniques enable robots to plan and execute long-horizon tasks by abstracting the search space and using human-like instructions to guide complex manipulations.

The contributions of this thesis include the development of novel algorithms for soft-body manipulation, hierarchical policy modeling, and the integration of generative models with reinforcement learning. These advancements offer new insights into the relationship between learning, modeling, and optimization in robotics.