ISSTA 2019
Mon 15 - Fri 19 July 2019 Beijing, China
Thu 18 Jul 2019 14:00 - 14:22 at Grand Ballroom - Testing and Machine Learning Chair(s): Hongyu Zhang

The past decade has seen the great potential of applying deep neural network (DNN) based software to safety-critical scenarios, such as autonomous driving. Similar to traditional software, DNNs could exhibit incorrect behaviors, caused by hidden defects, leading to severe accidents and losses. In this paper, we propose DeepHunter, a coverage-guided fuzz testing framework for detecting potential defects of general-purpose DNNs. To this end, we first propose a new metamorphic mutation strategy to generate new semantically preserved tests, and leverage multiple extensible coverage criteria as feedback to guide the test generation. We further propose a new seed selection strategy that combines both diversity-based seed selection and recency-based seed selection. We finally implement 5 existing testing criteria and 4 seed selection strategies in DeepHunter. Large-scale experiments demonstrate that (1) the metamorphic mutation strategy is useful to generate new valid tests with the same semantics as the original seed, by a 98% validity ratio; (2) diversity-based seed selection is more important than recency-based seed selection in boosting the coverage and in detecting defects; (3) DeepHunter significantly outperforms the state of the art (TensorFuzz and DeepTest) by coverage as well as the quantity and diversity of defects identified; (4) using corner-region based criteria, DeepHunter tends to be more useful to capture defects (capture 4x more defects than TensorFuzz in MobileNet) during the DNN quantization for platform migration.

Thu 18 Jul

issta-2019-Technical-Papers
14:00 - 15:30: Technical Papers - Testing and Machine Learning at Grand Ballroom
Chair(s): Hongyu ZhangThe University of Newcastle
issta-2019-Technical-Papers14:00 - 14:22
Talk
Xiaofei XieNanyang Technological University, Lei MaKyushu University, Felix Juefei-XuCarnegie Mellon University, Minhui Xue, Hongxu ChenNanyang Technological University, Yang LiuNanyang Technological University, Singapore, Jianjun ZhaoKyushu University, Bo LiUIUC, Jianxiong YinNVIDIA AI Tech Centre, Simon SeeNVIDIA AI Tech Centre
issta-2019-Technical-Papers14:22 - 14:45
Talk
Maxime CordySnT, University of Luxembourg, Steve Mullerunaffiliated, Mike PapadakisUniversity of Luxembourg, Yves Le TraonUniversity of Luxembourg
issta-2019-Technical-Papers14:45 - 15:07
Talk
Xia LiUniversity of Texas at Dallas, USA, Wei LiSouthern University of Science and Technology, Yuqun ZhangSouthern University of Science and Technology, Lingming Zhang
issta-2019-Technical-Papers15:07 - 15:30
Talk
Goran PiskachevFraunhofer IEM, Lisa Nguyen Quang DoPaderborn University, Eric BoddenHeinz Nixdorf Institut, Paderborn University and Fraunhofer IEM
DOI Pre-print Media Attached File Attached