UC Irvine

Autonomous Driving Systems (ADSes) are complex, safety-critical software systems whose correctness is essential to ensuring safe operation in the real world. Despite significant advances in simulation-based testing, existing approaches often struggle to effectively reveal ADS faults due to challenges such as ambiguous responsibility attribution, inherent system non-determinism, and limited test coverage. To address these limitations, this dissertation develops a systematic approach for more rigorous and reliable ADS testing. First, to address ambiguous fault attribution in scenario-based testing, this work introduces doppelganger testing, in which all vehicles in a simulated environment are controlled by instances of the same ADS. This setup eliminates external rule-violating actors, ensuring that observed safety violations can be attributed to the ADS under test and enabling consistent discovery of bug-revealing scenarios. Second, to address non-deterministic (flaky) testing in ADS, this work introduces deterministic frame-based testing, in which system-level execution traces are used to construct reproducible module-level tests. By reconstructing inputs to the planning module from recorded message-bus communications, this approach enables deterministic reproduction of system-level executions at the module level. Finally, to address limited test coverage in ADS testing, this work introduces a white-box scenario generation method that statically analyzes the ADS implementation to identify conditions required to activate distinct planning behaviors. It then uses belief-guided evolutionary search to allocate testing effort across scenarios targeting these behaviors. This approach expands coverage of the planning implementation and exposes failures that obstacle-centric black-box scenario generation misses.