UC Berkeley

The advent of advanced computational technologies and artificial intelligence has ushered in a new era of complex systems and applications, notably in the realms of autonomous vehicles (AVs) and robotics. These systems are increasingly required to make decisions autonomously in dynamic and uncertain environments. Reinforcement Learning (RL) has emerged as a pivotal technique in this context, offering a framework for learning optimal decision-making strategies through interactions with the environment. However, ensuring safety and trustworthiness in these decisions remains a critical challenge, especially in safety-critical applications such as autonomous driving.

This dissertation addresses the aforementioned challenge by proposing innovative RL-based approaches, and is structured into three distinct but interconnected parts, each focusing on a unique aspect of RL in the context of safe and trustworthy decision-making.The thread of this dissertation is based on the exploration and advancement of RL techniques to ensure safety and reliability in autonomous decision-making systems, particularly in complex, dynamic environments.

We first establish the foundational aspects of RL in decision-making, particularly in uncertain and dynamic environments. The focus here is on enhancing RL to deal with real-world complexities, such as interacting with unpredictable agents, e.g., human drivers in AV scenarios, and handling distributional shifts in offline RL settings. This sets the stage for understanding and improving the decision-making capabilities of autonomous systems under uncertainty.

Building on the first part, we then explore the integration of hierarchical planning with RL. The emphasis is on creating frameworks that combine different levels of decision-making, balancing immediate, low-level safety concerns with high-level strategic objectives. The approach aims to address the limitations of traditional RL in complex, multi-agent environments and long-duration tasks, demonstrating improved adaptability and efficiency in real-time decision-making.

The final part represents a forward-looking approach to RL, focusing on the integration of offline and online learning methodologies. This part addresses the challenge of training RL agents in a manner that is both safe and effective, particularly in contexts where exploration can be costly or dangerous. By combining the strengths of large-scale offline data (expert demonstrations) with online learning, we present a novel framework for enhancing the safety and performance of RL agents in practical, real-world applications.