UC Berkeley

In the current landscape, where robotic manipulators are increasingly pivotal in both customized production and household services, this dissertation explores the advancement of skills and dexterity in robotic manipulation. It introduces an innovative paradigm that synergizes physical principles, especially contact mechanics, with robot learning. This approach aims to surpass the limitations of current learning frameworks that rely excessively on substantial datasets and intensive exploration. The dissertation unfolds in three aspects of contact-aware robot learning. Initially, it establishes robust and sample-efficient grasping for contact point planning. A contrastive grasp planning module is developed to counteract camera noise and bridge the simulation-to-reality gap, enhancing grasp robustness. This is followed by the introduction of the Maximum Likelihood Grasp Sampling Loss, which remarkably reduces training sample requirements by eightfold compared to conventional methods. The dissertation further ventures into the realm of multi-fingered hand manipulation, significantly broadening the capabilities of robotic manipulators. The second dimension integrates contact planning into diverse manipulation tasks. A methodology for contact-aware learning from demonstrations is proposed, enabling robots to acquire skills by mimicking human demonstrations, thereby streamlining the process of robotic skill acquisition. This is complemented by an exploration of safe contact strategies in robotic operations, focusing on maintaining operational safety while allowing interactions with environmental obstacles. Additionally, an intelligent robotic assembly framework is introduced, which amalgamates sequence reasoning transformers with meticulous contact point planning, delineating multi-level reasoning. In the third dimension, the dissertation addresses the sensing of contact through vision-based tactile sensors. It presents a technique for reconstructing contact profiles from sensor-derived image imprints, offering essential feedback during manipulation tasks. Collectively, these strands of research coalesce into a comprehensive suite of robot learning for grasping, manipulation, and sensing. These strategies aim to minimize the reliance on extensive manual engineering by contact models, thereby augmenting the efficiency and stability of robotic systems across various applications. This dissertation not only underscores the feasibility and advantages of integrating contact mechanics into robot learning but also substantiates the efficacy and practical applicability of these approaches through rigorous validations via simulations and real-world experiments with diverse manipulators and hands.