UC San Diego

Preventable patient harm is the root cause of many adverse events in healthcare, and is a leading cause of mortality and morbidity worldwide. One way to address this is through career-long clinical education, often via the use of robotic patient simulator (RPS) systems. These highly realistic human-like physical or virtual platforms enable clinical learners to safely practice their diagnostic, procedural, and social interaction skills without harming real patients. However, most commercial RPS systems lack a realistic depiction of non-verbal facial cues, limiting learner engagement and immersion, which can ultimately lead to incorrect skill transfer and patient harm.

In my PhD, I have worked to address this gap by building new interactive and expressive RPS systems, whose faces are based entirely on real patients, and the system's expressions are realistic. In this dissertation, I will describe the main contributions of my work, including 1) Developing an end-to-end analysis-modeling-synthesis framework that can easily and robustly recognize, model, and synthesize patient-driven facial expressions and clinical cues on the faces of virtual and physical RPS systems, 2) Developing new methods to create accurate computational models of multiple pathologies, including stroke and Bell's Palsy, 3) successfully synthesizing these models on RPS systems, and, 4) designing the RPS as a clinical educational tool tested with clinicians.

My research opens new avenues of exploration in robotics, human-robot interaction, and health technology, and may trigger a new round of relevant technological innovations by creating the next generation of RPS technology. My work will enable roboticists to discover platform-independent methods to control the facial expressions of both robots and virtual agents, yielding new modalities for interaction. Furthermore, disseminating the results of this work to the research community will help both the broader robotics and healthcare communities employ these novel systems in their own application domains. This work serves as a bridge between robotics and healthcare research and practice, and offers promising opportunities to reduce misdiagnoses and bias in healthcare, ultimately reducing preventable patient harm.