The design of assistive technology has traditionally focused on addressing the functional limitations associated with physical disabilities, often overlooking the holistic experience of using an object, tool, or application. This dissertation introduces a novel conceptual design space for accessible human-computer interaction that expands beyond this utilitarian approach. By integrating interaction, sensation, and user experience, each influenced by the dynamic relationship between user goals and context, this design space offers a comprehensive perspective for creating more inclusive and empowering technologies for people with disabilities, especially motor impairments.

My research contributes to this multidimensional design space through a series of interconnected studies, each contributing to different dimensions while showcasing their interdependencies: 1) On the interaction component, I present a hands-free video game controller based on facial expressions that allows individuals with motor impairments to play independently. This system not only provides users with severe motor impairments a novel input method that accommodates their current level of physical constraint but also enhances the overall gaming experience and social connectivity. I further explore AI-generated virtual companions and text-to-image generation tools, developing accessible interfaces for people with motor impairments through leveraging large language models. I demonstrate how advanced AI can be utilized to create more natural and engaging interfaces, ultimately empowering individuals with motor impairments to express their creativity in a more natural and effortless manner. 2) For the sensation component, I present a system that provides haptic feedback to simulate clicking a computer mouse for users with severe hand motor impairments. This study demonstrates how sensory feedback can be provided through alternative means, emphasizing the tight coupling between input actions and output responses in creating a sense of agency and control. In fact, this study shows the fundamental reason why holistic inclusion is critical in technology design. Importantly, this research study highlights the fundamental need for holistic inclusion in technology design. 3) The user experience component is exemplified by our study on simulating walking in virtual reality for a lifelong wheelchair user. This work provides crucial insights into designing immersive experiences that align with the mental models and emotional needs of users who have never walked, catering to unfulfilled desires for exploration and freedom of movement. Methodologically, my research is anchored in the domains of human-computer interaction and accessibility studies. I employ an iterative and participatory design process that actively engages diverse stakeholders, including end users with motor impairments and technology designers. My evaluation methods include user experiments, in-depth interviews, and diary studies each tailored to the specific requirements of individual projects.

My holistic design space reveals important synergies and trade-offs across the three components. For example, the choice of interaction method affects the type of experience that can be created, which in turn influences the appropriate sensory feedback. Similarly, focusing on the desired user experience guides decisions about the interaction modality and corresponding sensation design. This dissertation contributes to a more comprehensive understanding of accessible interaction design. By considering interaction, sensation, and user experience as part of an integrated whole, my goal is to open new possibilities for creating technologies that not only overcome physical limitations but also enhance independence, creativity, and quality of life for people with motor impairments.

The present investigation aims to identify best practices to design virtual reality (VR) story-worlds to promote prejudice reduction and foster prosocial attitudes toward under-represented groups. Past research shows that VR, which allows the simulation of scenarios that would otherwise be prohibitively difficult or impossible in real life, can be an effective tool for understanding human cognition and researching perception, decision-making, and behavior (Groom, Bailenson, Nass, 2009). In tightly controlled immersive digital simulations, one can inhabit an avatar’s digital body with salient features (different skin color, age, gender, social status) to trigger beliefs, affect, and behaviors different from normal day-to-day life (Bailenson &Yee, 2007). VR perspective-taking experiences focused on imagined intergroup contact with individuals from marginalized groups can increase prosocial behavior toward them (van Loon, Bailenson, Zaki, Bostick, Willer 2018). Intergroup contact theory hypothesizes that reducing anxiety, which is the cause of increased stereotyping against the outgroup, and permeating the social encounter with positive emotions (Miller, Smith, & Mackie, 2004) leads to prejudice mitigation (Pettigrew & Tropp, 2006). Beyond these insights from social psychology, sparse literature has explored how to design immersive story-worlds to instill prosocial attitudes and behaviors, including the effectiveness of employing photorealistic techniques. This research fills this gap and challenges the assumption that human perceptions inside virtual and physical worlds are equal if digital assets are photorealistic. By creating a taxonomy of design strategies for inclusive VR, displaying data gathered during playtesting six state-of-the-art VR experiences, it identifies which affordances and methodologies are significant to inducing compassion and prosocial attitudes in immersive social encounters and contributes a pragmatic approach to the design of VR for bias mitigation while considering the craft, ethical and humanistic dimensions of the medium.

Immersive technologies are widely used in interactive media art and storytelling today. A high-quality virtual environment and avatar can provide the audience with a strong sense of presence and ownership of the virtual body, evoking empathetic outcomes from the experience. However, the majority of immersive art we encounter today shows a strong favoring of visual and auditory sensation over interactions. They provide the audience with a limited number of ways to participate in an immersive narrative, leaving them a sense of being an “outsider” that breaks the connection between the audience’s virtual avatar and the virtual environment. Starting with Nicolas Bourriaud’s relational aesthetics for art creation and taking human relations and their social context as the main conceptual and practical point of departure for experience design, I develop artistic and technical strategies for creating immersive experiences that place the audience at the center to let them activate the narrative content and cultural meanings. My approach is to provide the user novel ways to relate themselves to what they see on a screen, and to allow for direct participation and engagement. Through an increased level of interactivity, co-presence and plausibility in immersive environments, my work aims to help the audience develop new relationships with other people, with virtual entities, and with AI agents. The hybrid systems introduced in this thesis describe novel ways of creating relational experiences by techniques learned from different domains such as soft robotics, quantum computing and social science. I also demonstrate authoring interface design that can be more widely adopted by the creative community to overcome the challenges often faced in immersive and interactive narrative compositions.