ISSTA 2022
Mon 18 - Fri 22 July 2022 Online
Fri 22 Jul 2022 00:20 - 00:40 at ISSTA 2 - Session 1-10: Neural Networks, Learning, NLP A
Fri 22 Jul 2022 15:40 - 16:00 at ISSTA 2 - Session 3-10: Neural Networks, Learning, NLP C

Test Input Prioritizers (TIP) for Deep Neural Networks (DNN) are an important technique to handle the typically very large test datasets efficiently, saving computation and labelling costs. This is particularly true for large scale, deployed systems, where inputs observed in production are recorded to serve as potential test or training data for next versions of the system. Feng et. al. propose DeepGini, a very fast and simple TIP and show that it outperforms more elaborate techniques such as neuron- and surprise coverage. In a large-scale study (4 case studies, 8 test datasets, 32’200 trained models) we verify their findings. However, we also find that other comparable or even simpler baselines from the field of uncertainty quantification, such as the predicted softmax likelihood or the entropy of the predicted softmax likelihoods perform equally well as DeepGini.

Fri 22 Jul

Displayed time zone: Seoul change

00:00 - 01:00
Session 1-10: Neural Networks, Learning, NLP ATechnical Papers at ISSTA 2
00:00
20m
Talk
AEON: A Method for Automatic Evaluation of NLP Test Cases
Technical Papers
Jen-tse Huang The Chinese University of Hong Kong, Jianping Zhang The Chinese University of Hong Kong, Wenxuan Wang The Chinese University of Hong Kong, Pinjia He The Chinese University of Hong Kong, Shenzhen, Yuxin Su Sun Yat-sen University, Michael Lyu The Chinese University of Hong Kong
DOI
00:20
20m
Talk
Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)
Technical Papers
Michael Weiss Università della Svizzera italiana (USI), Paolo Tonella USI Lugano
DOI Pre-print
00:40
20m
Talk
ε-weakened Robustness of Deep Neural Networks
Technical Papers
Pei Huang State Key Laboratory of Computer Science, Institution of Software, Chinese Academy of Sciences, Yuting Yang Institute of Computing Technology,Chinese Academy of Sciences; University of Chinese Academy of Sciences, Minghao Liu Institute of Software, Chinese Academy of Sciences, Fuqi Jia State Key Laboratory of Computer Science, Institution of Software, Chinese Academy of Sciences, Feifei Ma Institute of Software, Chinese Academy of Sciences, Jian Zhang Institute of Software at Chinese Academy of Sciences, China
DOI
15:00 - 16:20
Session 3-10: Neural Networks, Learning, NLP CTechnical Papers at ISSTA 2
15:00
20m
Talk
ASRTest: Automated Testing for Deep-Neural-Network-Driven Speech Recognition Systems
Technical Papers
Pin Ji Nanjing University, Yang Feng Nanjing University, Jia Liu Nanjing University, Zhihong Zhao Nanjing Tech Unniversity, Zhenyu Chen Nanjing University
DOI
15:20
20m
Talk
BET: Black-box Efficient Testing for Convolutional Neural Networks
Technical Papers
Wang Jialai Tsinghua University, Han Qiu Tsinghua University, Yi Rong Tsinghua University, Hengkai Ye Purdue University, Qi Li Tsinghua University, Zongpeng Li Tsinghua University, Chao Zhang Tsinghua University
DOI
15:40
20m
Talk
Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)
Technical Papers
Michael Weiss Università della Svizzera italiana (USI), Paolo Tonella USI Lugano
DOI Pre-print
16:00
20m
Talk
ε-weakened Robustness of Deep Neural Networks
Technical Papers
Pei Huang State Key Laboratory of Computer Science, Institution of Software, Chinese Academy of Sciences, Yuting Yang Institute of Computing Technology,Chinese Academy of Sciences; University of Chinese Academy of Sciences, Minghao Liu Institute of Software, Chinese Academy of Sciences, Fuqi Jia State Key Laboratory of Computer Science, Institution of Software, Chinese Academy of Sciences, Feifei Ma Institute of Software, Chinese Academy of Sciences, Jian Zhang Institute of Software at Chinese Academy of Sciences, China
DOI