UC San Diego

The creation of home robots that can aid human beings in daily mundane chores has been a long-standing goal of robotics research. Some common indoor tasks that service robots can help with include retrieving objects from different locations, observing and monitoring home environments, and rearranging household objects. Enabling artificial agents to perform such activities requires knowledge gathered from several broad topics in Embodied Artificial Intelligence (EAI), such as localization and mapping, contextual scene understanding, and efficient interaction strategies in realistic environments. A common approach adopted by existing methods is to create dense geometric representations of the environment, typically in the form of point-cloud reconstruction of indoor regions. However, there are two fundamental problems associated with such metric maps, (i) they are non-trivial to construct and often require constant updates as the surrounding world evolves over time, and (ii), they lack semantic information, thereby making data association and contextual indexing more challenging. As such, planning algorithms that utilize only these representations rarely generalize to complex tasks such as searching for objects in noisy real-world environments. The primary objective for this work has been to develop appropriate semantic models of the world to enable robots to make smart, and robust decisions while solving complex indoor tasks. The first component here is a hierarchical semantic representation of the world. Different levels in this structure can correspond to a variation in granularity of scene understanding - ranging from metric information to reasoning-based context, and topological layout. Given this hierarchical representation, the next step is to formulate a smart planning strategy that can adaptively extract only the necessary context for a particular object-interaction task. The combination of this semantic representation and planning module results in a hybrid semantic model that is inspired by human-level cognitive models, and their ability to generalize across domains. Several methods to estimate contextual models for inferring scene geometry and semantics have been presented in this dissertation, with applications in visual place categorization, object-goal visual navigation, and complete home robot rearrangement tasks. Finally, some of the existing challenges in this domain are mentioned, along with a few future directions for home robotics research.