Dynasto: Validity-Aware Dynamic–Static Parameter Optimization for Autonomous Driving TestingDistinguished Paper Award
Extensive simulation-based testing is important for assuring the safety of autonomous driving systems (ADS). However, generating safety-critical traffic scenarios remains challenging because failures often arise from rare, complex interactions with surrounding vehicles. Existing automatic scenario-generation approaches frequently fail to distinguish genuine ADS faults from collisions caused by implausible or invalid adversarial behaviors, and they typically optimize either scenario initialization or agent behavior in isolation.
We propose Dynasto, a two-step testing approach that jointly optimizes initial scenario parameters and dynamic adversarial behaviors to uncover realistic safety-critical failures. First, we train an adversarial agent using reinforcement learning (RL) with temporal-logic–based validity criteria and a safe-distance model inspired by ISO 34502 to promote behaviorally plausible failures. Second, a genetic algorithm (GA) searches over initial conditions while replaying the adversary’s failure-inducing behaviors to reveal additional failures that the RL agent alone does not uncover. Finally, a graph-based clustering pipeline groups failures into representative modes based on semantic event sequences.
Our evaluation experiments in HighwayEnv across two ADS controllers show that Dynasto finds 60%–70% more valid failures than an RL-only adversary under the same evaluation budget. With clustering, we obtain about 12 interpretable failure modes per system under test, revealing valid failures driven by weaknesses in ego-controller behavior. These results indicate that coordinated dynamic-static optimization with explicit validity constraints is effective for exposing safety-relevant failures in ADS testing.
Tue 19 MayDisplayed time zone: Seoul change
11:00 - 12:30 | Autonomous Systems & Robotics TestingIndustry / Research Papers at Room 101 Chair(s): Khouloud Gaaloul University of Michigan - Dearborn | ||
11:00 25mTalk | Dynasto: Validity-Aware Dynamic–Static Parameter Optimization for Autonomous Driving TestingDistinguished Paper Award Research Papers Dmytro Humeniuk Polytechnique Montréal, Mohammad Hamdaqa Polytechnique Montreal, Houssem Ben Braiek Polytechnique Montreal, Amel Bennaceur The Open University, UK, Foutse Khomh Polytechnique Montréal | ||
11:25 25mTalk | Natural Adversaries: Fuzzing Autonomous Vehicles with Realistic Roadside Object Placements Research Papers Yang Sun Singapore Management University, Haoyu Wang School of Computing and Information Systems, Singapore Management University, Chris Poskitt Singapore Management University, Jun Sun Singapore Management University DOI Pre-print | ||
11:50 15mTalk | Metamorphic Testing of Vision-Language Action–Enabled Robots Research Papers Pablo Valle Mondragon University, Sergio Segura SCORE Lab, I3US Institute, Universidad de Sevilla, Seville, Spain, Shaukat Ali Simula Research Laboratory and Oslo Metropolitan University, Aitor Arrieta Mondragon University Pre-print | ||
12:05 25mTalk | Assessing Vision–Language Models for Perception in Autonomous Underwater Robotic Software Industry Muhammad Yousaf Simula Research Laboratory, Aitor Arrieta Mondragon University, Shaukat Ali Simula Research Laboratory and Oslo Metropolitan University, Paolo Arcaini National Institute of Informatics, Shuai Wang DNV AS | ||