ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland
Mon 17 Apr 2023 16:00 - 16:20 at Grand canal - Session 5: Testing AI/ML systems Chair(s): Jie M. Zhang

Neural networks have been widely applied for performing tasks in critical domains, such as, for example, the medical domain; their robustness is, therefore, important to be guaranteed. In this paper, we propose a robustness definition for neural networks used for regression, by tackling some of the problems of existing robustness definitions. First of all, by following recent works done for classification problems, we propose to define the robustness of networks used for regression w.r.t. alterations of their input data that can happen in reality. Since different alteration levels are not always equally probable, the robustness definition is parameterized with the probability distribution of the alterations. The error done by this type of networks is quantifiable as the difference between the estimated value and the expected value; since not all the errors are equally critical, the robustness definition is also parameterized with a “tolerance” function that specifies how the error is tolerated. The current work has been motivated by the collaboration with the industrial partner that has implemented a medical sensor employing a Multilayer Perceptron for the estimation of the blood oxygen pressure. After having computed the robustness for the case study, we have successfully applied three techniques to improve the network robustness: data augmentation with recombined data, data augmentation with altered data, and incremental learning. All the techniques have proved to contribute to increasing the robustness, though in different ways.

Mon 17 Apr

Displayed time zone: Dublin change

16:00 - 18:00
Session 5: Testing AI/ML systemsResearch Papers / Previous Editions at Grand canal
Chair(s): Jie M. Zhang King's College London
16:00
20m
Talk
Robustness assessment and improvement of a neural network for blood oxygen pressure estimation
Previous Editions
Paolo Arcaini National Institute of Informatics , Andrea Bombarda University of Bergamo, Silvia Bonfanti University of Bergamo, Angelo Gargantini University of Bergamo, Daniele Gamba AISent S.r.l., Rita Pedercini AISent S.r.l.
DOI
16:20
20m
Talk
An Empirical Evaluation of Mutation Operators for Deep Learning Systems
Previous Editions
Gunel Jahangirova King's College London, Paolo Tonella USI Lugano
DOI
16:40
20m
Talk
Distributed Repair of Deep Neural Networks
Research Papers
Davide Li Calsi Politecnico di Milano, Matias Duran National Institute of Informatics, Xiao-Yi Zhang School of Computer and Communication Engineering, University of Science and Technology Beijing, Paolo Arcaini National Institute of Informatics , Fuyuki Ishikawa National Institute of Informatics
17:00
20m
Talk
Mutation Testing of Deep Reinforcement Learning Based on Real Faults
Research Papers
Florian Tambon Polytechnique Montréal, Vahid Majdinasab Polytechnique Montréal, Amin Nikanjam École Polytechnique de Montréal, Foutse Khomh Polytechnique Montréal, Giuliano Antoniol Polytechnique Montréal
Pre-print
17:20
20m
Talk
Repairing DNN Architecture: Are We There Yet?
Research Papers
Jinhan Kim KAIST, Nargiz Humbatova USI Lugano, Gunel Jahangirova King's College London, Paolo Tonella USI Lugano, Shin Yoo KAIST
Pre-print