UC Berkeley

Robots are increasingly deployed in environments that involve close interaction with humans. To operate safely and intelligently in these settings, they must handle diverse and reactive human behavior. However, collecting such data at scale is costly and limited. Moreover, real-world data provides only passive observations and does not capture how humans would respond under different robot actions. While simulation offers a promising alternative, existing approaches struggle to capture realistic human behavior, support counterfactual reasoning, and scale to complex interactive settings.We begin by quantifying human social preferences from real-world interaction data using a data-driven measure of courtesy that captures the influence of an agent’s behavior on others. This enables socially controllable agent behaviors ranging from courteous to aggressive interactions to be learned directly from data. Importantly, we observe that courteous behavior manifests differently across interaction contexts. Second, we address the limited coverage of long-tail and safety-critical interactions in real-world datasets through controllable generative simulation. We propose a guided and partial diffusion framework for controllable generation of safety-critical interactions with diverse aggressiveness levels and collision types. Furthermore, we integrate natural language conditioning into joint multi-agent diffusion models, enabling high-level specification of interactive behaviors for counterfactual simulation. Finally, we address the scalability of closed-loop simulation, where large generative behavior models are often computationally expensive at inference time. We introduce a self-play reinforcement learning framework that uses pretrained tokenized reference models to guide policy learning toward human-like behavior. This enables scalable and realistic multi-agent simulation while achieving up to an order-of-magnitude faster inference than large generative models.Together, this dissertation lays a practical foundation for realistic and controllable simulation of human behavior, enabling autonomous systems to reason about the counterfactual consequences of different actions in human-centered environments.