UC Berkeley

In this thesis, we study how to efficiently learn robotic manipulation skills from various perspectives. We start by reviewing the current trend of applying learning-based methods in the domain of robotics; despite their great potentials, these methods still have limits on sample efficiency and the ability to generalize across domains. We aim to tackle these issues from roughly two aspects: introducing suitable policy priors into the learning process and robotic specific representations that are effective and generalizable.

We first introduce an effective algorithm called residual reinforcement learning, which essentially regularizes policy learning in function space by superimposing its action with the control signal from a ``prior controller", the variance during policy learning is reduced at the cost of injecting certain amount of bias which can be quantified. We validate our method on a variety of contact-rich fine-grained robotic manipulation tasks. On the other perspective, we study how robot learning could benefit from an effective representation. We introduce an action space representation for reinforcement learning algorithms: an operational space force controller. When a RL agent is combined with this operational force controller, it tends to learn substantially faster and perform better than other action space in several high-precision assembly tasks. We then propose a representation that explicitly models gripper features in grasping tasks, this allows us to collect data entirely in simulation and transfer to real-world without any fine-tuning nor loss of performance. We conduct large-scale real world experiments on a variety of objects and environments to validate this method. On a similar fashion, we extend a method called domain randomization by explicitly modelling robot rigid transformations in the task of pose estimation, which again trains entirely on synthetic data and follows a zero-shot transfer to real world. Results show that our method could achieve significantly better accuracy. Finally, we conclude that injecting prior information in suitable forms such as control bias, state representation, simulation model, etc., will dramatically increase the performance in a broad range of hard robotic problems; however these inductive bias will make our solution less general. We point future directions as to come up with principled way to design and inject such prior information to learning systems.