- Main
Embodied Intelligence from Autonomous Experience
- Lin, Toru
- Advisor(s): Malik, Jitendra
Abstract
Robots hold the promise of assisting humans in unstructured, everyday environments, yet today's robot systems remain far behind humans in sensorimotor control. They are either dexterous but brittle, or generalizable but clumsy. The current dominant approaches in robot learning rely heavily on scaling human demonstrations; this dissertation argues that such approaches are fundamentally limited by the embodiment gap between humans and robots, as well as by the scarcity of high-quality data, especially for dexterous manipulation. Instead, I advocate for a shift toward Embodied Intelligence from Autonomous Experience: a paradigm in which robots acquire increasingly dexterous and generalizable skills through active exploration and self-improvement. This dissertation presents three primary contributions toward making this paradigm practical. First, I introduce methods for learning dexterous priors from human guidance, including a low-latency teleoperation system and a visuotactile policy learning pipeline that help bridge the human–robot data gap. Second, I develop principled techniques for practical sim-to-real reinforcement learning, demonstrating that complex, contact-rich manipulation skills can be learned efficiently without massive compute or photorealistic simulation. Third, I present frameworks that unify and extend existing algorithmic components toward more capable real-world robot systems. Together, these contributions enable robots to perform fine-grained, contact-rich, long-horizon tasks with a level of dexterity and generalization previously difficult to achieve with either model-based or learning-based methods. More importantly, they demonstrate a path toward robot learning systems that not only learn from humans, but also improve through autonomous embodied experience.