UC Berkeley

Spatial Computing interfaces such as augmented reality (AR), virtual reality (VR), and mixed reality (MR) have become promising modalities for next-generation computing platforms. Along with its potential impact on various technological applications, spatial computing comes with spatial limitations itself. Such experiences are physically constrained by the geometry and semantics of the local user's environment where existing physical elements are present. Unlike 2D screens, where a rectangular screen region can host digital content with possible overlay, 3D environments are occupied with diverse physical obstacles and functional constraints. This results in complex and, many times, non-convex activity spaces available for virtual content augmentation. Target environments are not necessarily known to content developers, and hence the ability to deploy large-scale curated experiences that can adapt to a diverse set of user spaces is challenging. This limitation is elevated in remote telepresence scenarios, where identifying a common ground physically accessible for all participants can become difficult, especially if users are unaware of the spatial layout of other participants' physical environments.

Motivated by these spatial challenges, this dissertation works towards developing context-aware generative frameworks which enable large-scale deployment of adaptable spatial computing experiences for everyday users in diverse target environments. By introducing novel workflows to learn from examples as priors and utilizing spatial optimization methods, the systems developed in this dissertation address the spatial challenges in spatial computing in various applications of remote workplaces, multi-user telepresence, and curated experiences such as games and education. The contributions of this dissertation consist of solving two general sets of problems: 1) developing curated context-aware spatial experiences for large-scale deployment, which include a wide variety of diverse target spaces not known to the content developer; and 2) facilitating telepresence experiences, where participants are not aware of each other's local spaces due to the Telepresence Spatial Mapping Problem (TSPM), explained in this dissertation. The frameworks developed in this work can play a role in increasing the adoption of spatial computing interfaces in everyday environments, allowing developers to design and curate content in scale without knowing the target scene of the user itself. Moreover, the frameworks proposed here can potentially facilitate remote workplace practices and virtual collaborations by decreasing the spatial requirements for telepresence systems.