UCLA

Building an intelligent machine with human-like perception, interaction, learning, and reasoning remains a significant and challenging problem. Despite the recent remarkable progress in artificial intelligence, especially the deep learning techniques, we are still far from reaching this goal. Human intelligence exhibits unique advantages in learning to solve multiple tasks from limited data, acquiring skills and knowledge from interactions, learning efficiently with stages, and generalizing concepts to novel domains and environments. Merely combining individual algorithms without a human-centric architecture is hopeless for achieving such comprehensive capabilities.

In this dissertation, we study the human-like holistic understanding in 3D scenes, which is the most related scenario to the real world. The core idea is to imitate the human's capability in perception, interaction, learning, and reasoning for solving holistic tasks. We first propose a framework for human-centric 3D scene parsing, reconstruction, and synthesis, focusing on integrating imagined humans into the perception system for interpreting the underlying human activities and intentions beyond the pixels. Then we describe several works on human-centric interaction understanding, including the human-object interactions and human-human interactions. Finally, we imitate the human-like learning and reasoning abilities by studying how to learn concepts with curriculum, design efficient closed-loop neural-grammar-symbolic learning algorithm, and build a concept learning framework that achieves systematic generalization.