Write a Blog >>
ICSE 2020
Wed 24 June - Thu 16 July 2020
Tue 7 Jul 2020 08:05 - 08:17 at Goguryeo - I5-Deep Learning Testing and Debugging Chair(s): Pooyan Jamshidi

Deep learning (DL) applications are becoming increasingly popular. Their reliabilities largely depend on the performance of DL models integrated in these applications. Traditional techniques need to retrain the models or rebuild and redeploy the applications for coping with unexpected conditions beyond the models’ handling capabilities. In this paper, we take a fault tolerance approach, DISSECTOR, to distinguishing those inputs that represent unexpected conditions (beyond-inputs) from normal inputs that are still within the models’ handling capabilities (within-inputs), thus keeping the applications still function with expected reliabilities. The key insight of DISSECTOR is that a DL model should interpret a within-input with increasing confidence, while a beyond-input would probably cause confused guesses in the prediction process. DISSECTOR works in an application-specific way, adaptive to DL models used in applications, and extremely efficiently, scalable to large-size datasets from complex scenarios. The experimental evaluation shows that DISSECTOR outperformed state-of-the-art techniques in the effectiveness (AUC: avg. 0.8935 and up to 0.9894) and efficiency (runtime overhead: only 3.3–5.8 milliseconds). Besides, it also exhibited encouraging usefulness in defensing against adversarial inputs (AUC: avg. 0.9983) and improving a DL model’s actual accuracy in use (up to 16% for CIFAR-100 and 20% for ImageNet).

Tue 7 Jul

Displayed time zone: (UTC) Coordinated Universal Time change

08:05 - 09:05
I5-Deep Learning Testing and DebuggingTechnical Papers / Demonstrations at Goguryeo
Chair(s): Pooyan Jamshidi University of South Carolina
08:05
12m
Talk
DISSECTOR: Input Validation for Deep Learning Applications by Crossing-layer DissectionTechnical
Technical Papers
Huiyan Wang State Key Lab. for Novel Software Tech. and Dept. of Comp. Sci. and Tech., Nanjing University, Nanjing, China, Jingwei Xu Nanjing University, Chang Xu Nanjing University, Xiaoxing Ma Nanjing University, Jian Lu Nanjing University
08:17
12m
Talk
White-box Fairness Testing through Adversarial SamplingACM SIGSOFT Distinguished Paper AwardsTechnical
Technical Papers
Peixin Zhang Zhejiang University, Jingyi Wang National University of Singapore, Singapore, Jun Sun Singapore Management University, Guoliang Dong Computer College of Zhejiang University, Xinyu Wang Zhejiang University, Xingen Wang Zhejiang University, Jin Song Dong National University of Singapore, Dai Ting Huawei Corporation
08:29
3m
Talk
FeatureNET: Diversity-driven Generation of Deep Learning ModelsDemo
Demonstrations
Salah Ghamizi SntT - University of Luxembourg, Maxime Cordy SnT, University of Luxembourg, Mike Papadakis University of Luxembourg, Yves Le Traon University of Luxembourg
08:32
3m
Talk
EvalDNN: A Toolbox for Evaluating Deep Neural Network ModelsDemo
Demonstrations
Yongqiang TIAN The Hong Kong University of Science and Technology, Zhihua Zeng Zhejiang University, Ming Wen Huazhong University of Science and Technology, China, Yepang Liu Southern University of Science and Technology, Tzu-yang Kuo The Hong Kong University of Science and Technology, Shing-Chi Cheung Department of Computer Science and Engineering, The Hong Kong University of Science and Technology
08:35
12m
Talk
Taxonomy of Real Faults in Deep Learning SystemsACM SIGSOFT Distinguished Artifact AwardsArtifact ReusableTechnicalArtifact Available
Technical Papers
Nargiz Humbatova Università della Svizzera italiana, Gunel Jahangirova Università della Svizzera italiana, Gabriele Bavota Università della Svizzera italiana, Vincenzo Riccio Università della Svizzera italiana, Andrea Stocco Università della Svizzera italiana, Paolo Tonella Università della Svizzera italiana
08:47
12m
Talk
An Empirical Study on Program Failures of Deep Learning JobsACM SIGSOFT Distinguished Paper AwardsTechnical
Technical Papers
Ru Zhang Microsoft Research, Wencong Xiao Alibaba, Hongyu Zhang University of Newcastle, Australia, Yu Liu Microsoft Research, Haoxiang Lin Microsoft Research, Mao Yang Microsoft Research
DOI Pre-print