ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland
Mon 17 Apr 2023 16:40 - 17:00 at Grand canal - Session 5: Testing AI/ML systems Chair(s): Jie M. Zhang

Deep Neural Networks (DNNs) are applied in several safety-critical domains and their trustworthiness is of paramount importance. For example, DNNs used in autonomous driving as classifiers should not misclassify detected objects; however, since obtaining perfect accuracy is not possible, special attention should be given to the most critical cases, e.g., pedestrians. This has been confirmed by the consortium of our partners from the automotive domain that provided us with specific risk levels for different misclassifications. A recent approach to improve DNN performance is to localise DNN weights responsible for the misclassifications and then adjust (repair) them to improve the misclassifications. However, they under-perform when they need to consider multiple misclassifications, and they do not consider the risk levels of the different misclassifications. To tackle this, we propose DistrRep, a distributed repair approach that first finds the best fixes for each critical misclassification, and then integrates them in a single repaired DNN model, by considering the risk levels. We assess DistrRep over three DNN models and a dataset of autonomous driving images, by considering requirements specified by our industrial partners. Experiments show that DistrRep is more effective than baseline approaches based on retraining, and other risk-unaware repair approaches.

Mon 17 Apr

Displayed time zone: Dublin change

16:00 - 18:00
Session 5: Testing AI/ML systemsResearch Papers / Previous Editions at Grand canal
Chair(s): Jie M. Zhang King's College London
16:00
20m
Talk
Robustness assessment and improvement of a neural network for blood oxygen pressure estimation
Previous Editions
Paolo Arcaini National Institute of Informatics , Andrea Bombarda University of Bergamo, Silvia Bonfanti University of Bergamo, Angelo Gargantini University of Bergamo, Daniele Gamba AISent S.r.l., Rita Pedercini AISent S.r.l.
DOI
16:20
20m
Talk
An Empirical Evaluation of Mutation Operators for Deep Learning Systems
Previous Editions
Gunel Jahangirova King's College London, Paolo Tonella USI Lugano
DOI
16:40
20m
Talk
Distributed Repair of Deep Neural Networks
Research Papers
Davide Li Calsi Politecnico di Milano, Matias Duran National Institute of Informatics, Xiao-Yi Zhang School of Computer and Communication Engineering, University of Science and Technology Beijing, Paolo Arcaini National Institute of Informatics , Fuyuki Ishikawa National Institute of Informatics
17:00
20m
Talk
Mutation Testing of Deep Reinforcement Learning Based on Real Faults
Research Papers
Florian Tambon Polytechnique Montréal, Vahid Majdinasab Polytechnique Montréal, Amin Nikanjam École Polytechnique de Montréal, Foutse Khomh Polytechnique Montréal, Giuliano Antoniol Polytechnique Montréal
Pre-print
17:20
20m
Talk
Repairing DNN Architecture: Are We There Yet?
Research Papers
Jinhan Kim KAIST, Nargiz Humbatova USI Lugano, Gunel Jahangirova King's College London, Paolo Tonella USI Lugano, Shin Yoo KAIST
Pre-print