UC Riverside

Perceiving the environment is one of the most fundamental tasks to enabling driving automation, which is regarded as the revolutionary solution to addressing the safety, mobility, and sustainability issues of contemporary transportation systems. Although an unprecedented evolution is now happening in the area of computer vision for object perception, state-of-the-art perception methods are still struggling with sophisticated real-world traffic environments due to the inevitable physical occlusion and limited receptive field of single-vehicle perception systems. Based on multiple spatially separated perception nodes, Cooperative Perception has emerged to unlock the bottleneck of perception for driving automation.

In this dissertation, cooperative perception is explored from multiple perspectives for driving automation, including systematic architecture, heterogeneity, scalability, computation complexity, communication bandwidth, and real-world implementation challenges. Specifically, the main contribution of this dissertation can be divided into two categories: 1) the methodological innovation which provides a series of cooperative perception methods, considering different open-challenging tasks; and 2) the proof-of-concept systems development prototypes that cover different cooperative perception applications in both simulation and real-world environments.

For general cooperative perception topics, this dissertation proposes two pipelines: PillarGrid and VINet to address vehicle-to-infrastructure cooperative perception and lightweight, scalable, and heterogeneous cooperative perception, respectively. Furthermore, three methods have been developed that dive into the open-challenging tasks of cooperative perception, including the feature-sharing challenge, adaptive cooperation challenge, and efficient relative pose estimation challenge. A data acquisition platform, CARTI, has been developed and open-sourced for supporting training and testing for cooperative perception models. Quantitative and qualitative experiments have demonstrated the effectiveness and advantages of the proposed algorithms.

For the proof-of-concept, in this dissertation, three prototypes were developed for evaluating the different perspectives of cooperative perception: 1) a game engine-based co-simulation system, CMM CoSim, for cooperative perception for CAV applications; 2) a real-world roadside LiDAR-based system for infrastructure-based cooperative perception, and 3) a real-world multi-agent system, OpenCooper, for demonstrating cooperative perception in realistic conditions.