UC San Diego

Effective face-to-face communication with other people presents challenges for many reasons. We may be distracted, fatigued, or emotional and unable to communicate well or remember the details of a conversation. Certain contexts, such as medical and therapeutic interaction, present additional challenges to effective communication due to illness, stress, or disability. This thesis presents research on novel systems to support collocated interaction for populations who have communication challenges related to a hearing, speech, or developmental disability. I describe three research projects that involve understanding face-to-face communication needs, designing novel systems to augment interaction among collocated participants, and evaluating how these systems shape the nature of human-human interaction. First, the SIDES project introduces a cooperative tabletop computer game designed as a social skills therapy tool for children with Asperger's Syndrome, an Autism Spectrum Disorder. Children with AS often have difficulty understanding accepted social conventions, reading facial expressions, interpreting body language, and understanding social protocols. Findings indicate that cooperative tabletop computer games are a motivating and supportive tool for facilitating face-to-face interaction involving this population. Second, the Shared Speech Interface (SSI) project involves the design and evaluation of an application for an interactive multitouch tabletop display that facilitates medical communication between a Deaf patient and a hearing, non-signing physician. SSI provides Deaf individuals with a more private and independent alternative for medical communication. SSI also reshapes communication between the doctor and Deaf patient in important ways. Third, the Write-N-Speak project examines face-to-face communication for individuals with aphasia. Through a year-long field study, I understand the process of speech-language therapy for older adults with aphasia and introduce digital pen technology into this work environment. This project also involves the design and field deployment of Write-N-Speak, a programmable toolkit that allows non-technical end-users to independently create custom interactive paper materials. Through this thesis, I provide a deeper understanding of face-to-face human interaction involving critical user populations, such as children with autism, Deaf people, and older adults with aphasia. I introduce novel prototype systems that support face-to-face interaction among participants with varying abilities and examine how these systems augment collocated interaction