Understanding human behavior plays a significant role in many industrial applications, such as autonomous driving and robotics, that entail human participation. Having an efficacious human behavior model facilitates intelligent agents in arriving at high-quality decisions while aligning with human values. In the past decades, while we have witnessed the remarkable progress of human behavior modeling with modern deep learning approaches, it remains a challenging task due to the inherent stochasticity of human behavior and the intricate structural design of deep learning models.
This dissertation focuses on learning-based human behavior prediction and generation for autonomous driving and robotics from both the model and learning algorithm design perspective. In particular, the dissertation is concerned with two manifestations of human behaviors: human driving behavior and 3D human motion. The dissertation is divided into two parts. Part I focuses on improving the generalization of human driving behavior prediction problems in different problem settings. Chapter 2 introduces a transferable human driving behavior prediction model leveraging domain-knowledge-based representation. In Chapter 3, a neural memory system based on generative models is proposed to enable the capability of continual learning for multi-agent vehicle trajectory prediction. In Chapter 4, an uncertainty analysis of trajectory prediction in terms of static and dynamic information is provided, and a static-information-only-based approach for difficult instance discovery is proposed to improve the adaptability of the prediction model. Part II focuses on two properties of human behavior generation: diversity and feasibility. Chapter 5 introduces a novel generative model exploring diverse 3D human motion behavior and enables testing-time adjustment of prediction and generation. Chapter 6 introduces a differentiable safety-critical control framework to ensure the feasibility and generalizability of generated motions, and we demonstrate the efficiency of the proposed approach in the collision avoidance application.