Write a Blog >>
ISSTA 2020
Sat 18 - Wed 22 July 2020
Mon 20 Jul 2020 16:50 - 17:10 at Zoom - MACHINE LEARNING I Chair(s): Divya Gopinath

Deep neural networks have been deployed in many software systems to assist in various classification tasks. In the company with fantastic effectiveness in classification, DNNs could also exhibit incorrect behaviors and result in accidents and losses. Therefore, testing techniques that can detect incorrect DNN behaviors and improve DNN quality are extremely necessary and critical. However, the testing oracle, which defines the correct output for a given input, is often not available in the automated testing. To obtain the oracle information, the testing tasks of DNN-based systems usually require expensive human efforts to label the testing data, which significantly slows down the process of quality assurance.

To mitigate this problem, we propose DeepGini, a test prioritization technique designed based on a statistical perspective of DNN. Such a statistical perspective allows us to reduce the problem of measuring misclassification probability to the problem of measuring set impurity. DeepGini allows us to identify possibly-misclassified tests quickly. These tests are very useful in improving the robustness of DNNs. To evaluate our technique, we conduct an extensive empirical study on popular datasets and prevalent DNN models. The experiment results demonstrate that DeepGini outperforms the existing coverage-based techniques in prioritizing test cases regarding both effectiveness and efficiency. In addition, we observe that the tests prioritized at the front by DeepGini are more effective in improving the DNN quality in comparison with the coverage-based techniques.

Mon 20 Jul

Displayed time zone: Tijuana, Baja California change

16:10 - 17:10
MACHINE LEARNING ITechnical Papers at Zoom
Chair(s): Divya Gopinath NASA Ames (KBR Inc.)

Public Live Stream/Recording. Registered participants should join via the Zoom link distributed in Slack.

16:10
20m
Talk
Reinforcement Learning Based Curiosity-Driven Testing of Android ApplicationsACM SIGSOFT Distinguished Paper Award
Technical Papers
Minxue Pan Nanjing University, An Huang , Guoxin Wang , Tian Zhang Nanjing University, Xuandong Li Nanjing University
DOI Media Attached
16:30
20m
Talk
Effective White-Box Testing of Deep Neural Networks with Adaptive Neuron-Selection StrategyArtifacts Evaluated – ReusableArtifacts AvailableArtifacts Evaluated – FunctionalACM SIGSOFT Distinguished Paper Award
Technical Papers
Seokhyun Lee Korea University, South Korea, Sooyoung Cha Korea University, South Korea, Dain Lee , Hakjoo Oh Korea University, South Korea
DOI Media Attached
16:50
20m
Talk
DeepGini: Prioritizing Massive Tests to Enhance the Robustness of Deep Neural Networks
Technical Papers
Yang Feng Nanjing University, Qingkai Shi The Hong Kong University of Science and Technology, Xinyu Gao , Muhammed Kerem Kahraman , Chunrong Fang Nanjing University, Zhenyu Chen Nanjing University
DOI