Risk-Sensitive Safety Analysis and Control for Trustworthy Autonomy
- Author(s): Chapman, Margaret P
- Advisor(s): Tomlin, Claire
- et al.
Methods for managing dynamical systems typically invoke one of two perspectives. In the worst-case perspective, the system is assumed to behave in the most harmful way; this perspective is used to provide formal safety guarantees. In the risk-neutral perspective, the system is assumed to behave as expected; this perspective is invoked in reinforcement learning and stochastic optimal control. While the worst-case perspective is useful for safety analysis, it can lead to unnecessarily conservative decisions, especially in settings where uncertainties are non-adversarial. The risk-neutral perspective is less conservative and useful for optimizing the system's performance on average. However, optimizing average performance is not guaranteed to protect against harmful outcomes and thus is not appropriate for safety-critical applications.
This thesis consists of two parts. First, we present an analytical and computational toolkit for cancer modeling and management that we have developed with cancer biologists by invoking the worst-case perspective. In addition to providing biological insights about breast cancer and theoretical insights about switched systems, this work has motivated the need for new mathematical methods that facilitate less conservative but still protective control of dynamical systems.
Towards this aim, we have devised a risk-sensitive mathematical method for safety analysis that blends the worst-case and risk-neutral perspectives by leveraging the Conditional Value-at-Risk measure. The second part of this thesis presents the mathematical development of this risk-sensitive safety analysis method. We also show its practical application to evaluating the safety of urban water infrastructure, using a numerical example that has been developed in collaboration with water resources engineers.