UC Irvine

This dissertation addresses the pressing issue of clinician burnout linked to historical trends of excessive documentation workload and poor usability of health information technology. In recent years, digital scribe systems—ambient speech-based documentation systems that capture patient-clinician conversations to generate preliminary clinical notes—have emerged as a promising solution to reduce documentation burden. However, such patient- and clinician-facing technologies must overcome significant technical, social, and organizational challenges to achieve meaningful real-world impact.This research employs a mixed-methods approach structured around three primary research questions. First, a systematic literature review and an interview study with human medical scribes were conducted to understand the complexity of their roles and identify critical design considerations for digital scribes. Insights gained highlight the necessity for digital scribes to replicate not only transcription tasks but also information-foraging activities across diverse, non-standardized data sources, physical tasks, and context-driven applications of scribes' skills. Second, evaluation studies assessed the performance of contemporary automatic speech recognition (ASR) and speaker diarization models using simulated patient-clinician dialogues. Analyses concentrated on general performance metrics such as word error rate (WER) and word-diarization error rate (WDER), stratified by contextual factors, including specialized medical conversation models and non-lexical conversational sounds indicating agreement or disagreement. Results indicated improved ASR performance compared to previous studies; however, capturing nuanced conversational elements remains a substantial challenge. These findings underscore the need for refined models capable of reliably extracting medically pertinent information. Third, a qualitative comparison of recorded patient-clinician encounters with corresponding EHR notes was conducted to systematically analyze patterns of information inclusion, omission, and insertion relative to spoken dialogues. Results indicated clinicians selectively documented information deemed actionable, omitted emotionally expressive and casual conversational elements, and regularly inserted contextually relevant historical or regulatory data from templates or past records. These findings emphasize that clinical documentation is an interpretive, deliberate, and context-driven process rather than a straightforward summarization task. Collectively, these findings highlight that effective digital scribes require more than advanced ASR technology alone. Their design should consider seamless integration into clinical workflows, balance fidelity to spoken conversations with the structured and regulated demands of clinical documentation, and replicate physician-like contextual decision-making. By clarifying these complexities, this dissertation provides valuable insight and practical guidance to advance digital scribes technologies, ultimately aiming to reduce clinician documentation burdens and enhance patient care.