UC San Diego

Machine perception is essential for the functionality and safety of modern autonomous system, yet achieving accurate and robust estimations of the state of the environment from sensor data remains challenging. Core perception tasks like multi-object tracking (MOT) and simultaneous localization and mapping (SLAM) are frequently addressed using a two-stage paradigm, where sensor data is preprocessed into an intermediate representation before being used for inference. This paradigm, however, suffers from a critical weakness that the initial processing stage can discard valuable information, leading to suboptimal performance in challenging conditions. This dissertation posits that a graph-based Bayesian framework provides a powerful and flexible platform for overcoming these limitations. This framework leverages the explicit structure of factor graphs to pursue two powerful, complementary strategies: first, by designing more physically-grounded statistical models for direct inference on raw sensor data, and second, by seamlessly integrating data-driven learning components to correct and enhance simpler, hand-designed models. We demonstrate the potential of this approach through three primary contributions. We first address the dominant detect-then-track (DTT) paradigm by introducing a hybrid method that mitigates model mismatch, using a graph neural network (GNN) to learn corrections for a BP algorithm. This approach significantly enhances tracking performance in complex scenes by learning to reject false alarms and improve data association, as demonstrated on a large-scale autonomous driving dataset. Next, to overcome the fundamental information loss of DTT, we propose a principled track-before-detect (TBD) method for MOT, built upon a comprehensive signal model that operates directly on raw sensor data. Finally, we extend this direct-processing philosophy to a new domain, introducing the first multipath-based SLAM algorithm that functions on raw radio signals without a separate, lossy channel estimation stage. This enables precise geometric mapping in challenging indoor environments by resolving multipath components (MPCs) that are unresolvable by traditional two-stage methods. In each case, a factor graph is constructed and a scalable BP algorithm is developed to enable efficient inference. Through extensive validation on both synthetic and real data, our contributions collectively demonstrate that a graph-based approach leads to significant improvements in accuracy. This work provides a versatile foundation for the next generation of intelligent perception systems.