ICSE 2025
Sat 26 April - Sun 4 May 2025 Ottawa, Ontario, Canada
Fri 2 May 2025 15:15 - 15:30 at 215 - SE for AI with Quality 2 Chair(s): Romina Spalazzese

Simulation-based testing represents an important step to ensure the reliability of autonomous driving software. In practice, when companies rely on third-party general-purpose simulators, either for in-house or outsourced testing, the generalizability of testing results to real autonomous vehicles is at stake. In this paper, we enhance simulation-based testing by introducing the notion of digital siblings�a multi-simulator approach that tests a given autonomous vehicle on multiple general-purpose simulators built with different technologies, that operate collectively as an ensemble in the testing process. We exemplify our approach on a case study focused on testing the lane-keeping component of an autonomous vehicle. We use two open-source simulators as digital siblings, and we empirically compare such a multi-simulator approach against a digital twin of a physical scaled autonomous vehicle on a large set of test cases. Our approach requires generating and running test cases for each individual simulator, in the form of sequences of road points. Then, test cases are migrated between simulators, using feature maps to characterize the exercised driving conditions. Finally, the joint predicted failure probability is computed, and a failure is reported only in cases of agreement among the siblings. Our empirical evaluation shows that the ensemble failure predictor by the digital siblings is superior to each individual simulator at predicting the failures of the digital twin. We discuss the findings of our case study and detail how our approach can help researchers interested in automated testing of autonomous driving software.

Fri 2 May

Displayed time zone: Eastern Time (US & Canada) change

14:00 - 15:30
SE for AI with Quality 2Journal-first Papers at 215
Chair(s): Romina Spalazzese Malmö University
14:00
15m
Talk
Beyond Accuracy: An Empirical Study on Unit Testing in Open-source Deep Learning ProjectsSE for AI
Journal-first Papers
Han Wang Monash University, Sijia Yu Jilin University, Chunyang Chen TU Munich, Burak Turhan University of Oulu, Xiaodong Zhu Jilin University
14:15
15m
Talk
Boundary State Generation for Testing and Improvement of Autonomous Driving SystemsSE for AI
Journal-first Papers
Matteo Biagiola Università della Svizzera italiana, Paolo Tonella USI Lugano
DOI Pre-print
14:30
15m
Talk
D3: Differential Testing of Distributed Deep Learning with Model GenerationSE for AI
Journal-first Papers
Jiannan Wang Purdue University, Hung Viet Pham York University, Qi Li , Lin Tan Purdue University, Yu Guo Meta Inc., Adnan Aziz Meta Inc., Erik Meijer
14:45
15m
Talk
Evaluating the Impact of Flaky Simulators on Testing Autonomous Driving SystemsSE for AI
Journal-first Papers
Mohammad Hossein Amini University of Ottawa, Shervin Naseri University of Ottawa, Shiva Nejati University of Ottawa
15:00
15m
Talk
Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension StudySE for AI
Journal-first Papers
Luca Giamattei Università di Napoli Federico II, Matteo Biagiola Università della Svizzera italiana, Roberto Pietrantuono Università di Napoli Federico II, Stefano Russo Università di Napoli Federico II, Paolo Tonella USI Lugano
DOI Pre-print
15:15
15m
Talk
Two is Better Than One: Digital Siblings to Improve Autonomous Driving TestingSE for AI
Journal-first Papers
Matteo Biagiola Università della Svizzera italiana, Andrea Stocco Technical University of Munich, fortiss, Vincenzo Riccio University of Udine, Paolo Tonella USI Lugano
DOI Pre-print
Hide past events