TCSE logo 
 Sigsoft logo
Sustainability badge

This program is tentative and subject to change.

Fri 2 May 2025 15:00 - 15:15 at 215 - SE for AI with Quality 2

In a recent study, Reinforcement Learning (RL) used in combination with many-objective search, has been shown to outperform alternative techniques (random search and many-objective search) for online testing of Deep Neural Network-enabled systems. The empirical evaluation of these techniques was conducted on a state-of-the-art Autonomous Driving System (ADS). This work is a replication and extension of that empirical study. Our replication shows that RL does not outperform pure random test generation in a comparison conducted under the same settings of the original study, but with no confounding factor coming from the way collisions are measured. Our extension aims at eliminating some of the possible reasons for the poor performance of RL observed in our replication: (1) the presence of reward components providing contrasting feedback to the RL agent; (2) the usage of an RL algorithm (Q-learning) which requires discretization of an intrinsically continuous state space. Results show that our new RL agent is able to converge to an effective policy that outperforms random search. Results also highlight other possible improvements, which open to further investigations on how to best leverage RL for online ADS testing.

This program is tentative and subject to change.

Fri 2 May

Displayed time zone: Eastern Time (US & Canada) change

14:00 - 15:30
SE for AI with Quality 2Journal-first Papers at 215
14:00
15m
Talk
Beyond Accuracy: An Empirical Study on Unit Testing in Open-source Deep Learning Projects
Journal-first Papers
Han Wang Monash University, Sijia Yu Jilin University, Chunyang Chen TU Munich, Burak Turhan University of Oulu, Xiaodong Zhu Jilin University
14:15
15m
Talk
Boundary State Generation for Testing and Improvement of Autonomous Driving Systems
Journal-first Papers
Matteo Biagiola Università della Svizzera italiana, Paolo Tonella USI Lugano
14:30
15m
Talk
D3: Differential Testing of Distributed Deep Learning with Model Generation
Journal-first Papers
Jiannan Wang Purdue University, Hung Viet Pham York University, Qi Li , Lin Tan Purdue University, Yu Guo Meta Inc., Adnan Aziz Meta Inc., Erik Meijer
14:45
15m
Talk
Evaluating the Impact of Flaky Simulators on Testing Autonomous Driving Systems
Journal-first Papers
Mohammad Hossein Amini University of Ottawa, Shervin Naseri University of Ottawa, Shiva Nejati University of Ottawa
15:00
15m
Talk
Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study
Journal-first Papers
Luca Giamattei Università di Napoli Federico II, Matteo Biagiola Università della Svizzera italiana, Roberto Pietrantuono Università di Napoli Federico II, Stefano Russo Università di Napoli Federico II, Paolo Tonella USI Lugano
15:15
15m
Talk
Two is Better Than One: Digital Siblings to Improve Autonomous Driving Testing
Journal-first Papers
Matteo Biagiola Università della Svizzera italiana, Andrea Stocco Technical University of Munich, fortiss, Vincenzo Riccio University of Udine, Paolo Tonella USI Lugano
Pre-print
:
:
:
: