Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia
Thu 18 May 2023 11:30 - 11:45 at Meeting Room 102 - AI testing 1 Chair(s): Matthew B Dwyer

Regression errors of Deep Neural Network (DNN) models refer to the case that predictions were correct by the old-version model but wrong by the new-version model. They frequently occur when upgrading DNN models in production systems, causing disproportionate user experience degradation. In this paper, we propose a lightweight regression error reduction approach with two goals: 1) without requiring model retraining and even data, and 2) without sacrificing the accuracy. The proposed approach is built upon the key insight rooted in the unmanaged model uncertainty, which is intrinsic to DNN models but not well explored, especially in the context of quality assurance of DNN models. Specifically, we propose a simple yet effective ensemble strategy that estimates and aligns the two models’ uncertainty in each prediction. We show that a Pareto improvement that reduces the regression errors without compromising the overall accuracy can be both guaranteed in theory and largely achieved in practice. Comprehensive experiments with various representative models and datasets also confirm that our approaches significantly outperform the state-of-the-art alternatives.

Thu 18 May

Displayed time zone: Hobart change

11:00 - 12:30
11:00
15m
Talk
When and Why Test Generators for Deep Learning Produce Invalid Inputs: an Empirical Study
Technical Track
Vincenzo Riccio University of Udine, Paolo Tonella USI Lugano
Pre-print
11:15
15m
Talk
Fuzzing Automatic Differentiation in Deep-Learning Libraries
Technical Track
Chenyuan Yang University of Illinois at Urbana-Champaign, Yinlin Deng University of Illinois at Urbana-Champaign, Jiayi Yao The Chinese University of Hong Kong, Shenzhen, Yuxing Tu Huazhong University of Science and Technology, Hanchi Li University of Science and Technology of China, Lingming Zhang University of Illinois at Urbana-Champaign
11:30
15m
Talk
Lightweight Approaches to DNN Regression Error Reduction: An Uncertainty Alignment Perspective
Technical Track
Zenan Li Nanjing University, China, Maorun Zhang Nanjing University, China, Jingwei Xu , Yuan Yao Nanjing University, Chun Cao Nanjing University, Taolue Chen Birkbeck University of London, Xiaoxing Ma Nanjing University, Jian Lu Nanjing University
Pre-print
11:45
7m
Talk
DeepJudge: A Testing Framework for Copyright Protection of Deep Learning Models
DEMO - Demonstrations
Jialuo Chen Zhejiang University, Youcheng Sun The University of Manchester, Jingyi Wang Zhejiang University, Peng Cheng Zhejiang University, Xingjun Ma Deakin University
11:52
7m
Talk
DeepCrime: from Real Faults to Mutation Testing Tool for Deep Learning
DEMO - Demonstrations
Nargiz Humbatova USI Lugano, Gunel Jahangirova King's College London, Paolo Tonella USI Lugano
12:00
7m
Talk
DiverGet: a Search-Based Software Testing approach for Deep Neural Network Quantization assessment
Journal-First Papers
Ahmed Haj Yahmed École Polytechnique de Montréal, Houssem Ben Braiek École Polytechnique de Montréal, Foutse Khomh Polytechnique Montréal, Sonia Bouzidi National Institute of Applied Science and Technology, Rania Zaatour Potsdam Institute for Climate Impact Research
12:07
15m
Talk
Revisiting Neuron Coverage for DNN Testing: A Layer-Wise and Distribution-Aware Criterion
Technical Track
Yuanyuan Yuan The Hong Kong University of Science and Technology, Qi Pang HKUST, Shuai Wang Hong Kong University of Science and Technology