UC Irvine

With the increased adoption and use of smart home speakers in households, many young children have learned to use listening and speaking to interact with the voice assistants (VAs) for a variety of daily activities (e.g., asking questions, listening to music). Despite the proliferation of VAs, due to the technical limitations such as automatic speech recognition (ASR) and natural language understanding (NLU), children often experience communication breakdowns with VAs, limiting educational opportunities through these voice-based interfaces. Informed by prior literature on child-computer interaction, child language development, and audio and trivia game design, I first designed a trivia game Animal Actions using Amazon Alexa with interdisciplinary research teams. I then conducted an in-home user evaluation study with 18 children between 3 and 6 years old using a mobile phone. Through video analysis of children's gameplay, I examined children's interaction with Alexa from four areas: children's verbal behaviors, children's response errors, Alexa communication breakdowns, and children's communication repair strategies. I found that single-word responses were more accurately recognized and processed by Alexa than were multi-word phrase-level responses. When comparing children's response accuracy and errors, children's responses included a wide range of speech articulation, semantic, and syntactic differences leading to incidents of communication breakdowns, which were not accounted for during game design and game item development. Parent questionnaires also revealed similar usability issues with automatic speech recognition and natural language understanding when children use VAs at home for daily routines, and parents reported using VAs independently and collaboratively with their children to mitigate communication breakdowns. Based on these findings, I discuss sociotechnical, design, and ethical considerations to inform future research on designing accessible user interfaces in voice games to support children's communication skills, including those with communication disorders.