UC San Diego

Technology-mediated collaboration for physical tasks -- where a novice manipulates real objects in their space with the guidance of a remote expert -- helps extend expertise to remote and often resource-strapped environments. The spatial and immersive nature of emerging technologies like eXtended Reality (XR) promise to make this collaboration more natural and intuitive in ways that traditional screenbound collaborative systems can't. However, a plethora of challenges remain before it can fulfill this promise, and several key aspects of the design space are yet to be explored.

In this dissertation, I first introduce a framework for XR-mediated collaborative physical tasks that aims to serve as a cohesive resource for both the design of collaborator interfaces and the exploration of the unique capabilities spatial and immersive technology bring to remote collaboration. I then design Mixed and Virtual Reality interfaces for both experts and novices and evaluate them in realistic settings. These evaluations expand our understanding of XR as a collaborative medium and highlight the challenges associated with supporting the situational awareness experts need to partake in object-focused communication. An in-depth review of existing collaborative XR systems adds to the discourse and uncovers key design opportunities to enhance support for communication during physical tasks. This dissertation also highlights a core assumption within the field and challenges the need to fully support experts' environment awareness through a view of the task environment. I do this by leveraging the context-aware capabilities of the technology to provide alternate information representations and communication mechanisms that minimize the information foraging collaborators need to perform within the task environment to sustain object-focused communication. Controlled experiments evaluate the impact of these techniques on the collaborative experience and showcase how they reduce task load while increasing task performance and communication efficiency.

Put together, this work lays the foundations for rich collaborative experiences that go beyond the constraints of face-to-face interactions while making it easier and more sustainable to scale real-world expertise.