UC Berkeley

Simulation-based reinforcement learning has produced many of the most capable robot controllers reported in recent years, with policies trained in simulation and deployed zero-shot on legged, dexterous, and aerial systems. Reproducing this methodology on a new robot, however, is hard: the work spans an end-to-end pipeline whose stages depend on infrastructure of varying maturity and openness, and the closed or fragmented pieces of that infrastructure are the bottleneck more often than the algorithms running on top. This thesis develops the open infrastructure that each stage of the pipeline needs. For grounding, where a faithful simulated model of the hardware must exist before training is meaningful, we build a curated library of robot descriptions for an open physics engine and a system identification toolbox that calibrates those models against specific hardware. For training, where the iteration loop between rewards and resulting policies sets project velocity, we contribute open, GPU-accelerated reinforcement learning infrastructure ranging from a minimal-abstraction library for rapid prototyping to a manager-based framework for codebases that span multiple robots and task families, on which a single sim-to-real recipe transfers across six robot platforms within weeks. For task priors, the human-derived guidance needed when reward design alone is insufficient, we show that expert fingering annotations make dexterous bimanual piano playing learnable from sheet music, and we develop the inverse-kinematics infrastructure that converts human motion-capture, teleoperation, and video data into robot reference trajectories. The artifacts produced across these stages have been taken up by external research groups, robot manufacturers, and other simulator projects, and now serve as the substrate for downstream work on humanoid motion imitation, video-based piano playing, and teleoperated whole-body control. Across the chapters, the open, end-to-end infrastructure documented here is what allows the sim-to-real methodology to be reproduced across labs.