Write a Blog >>
ICSE 2021
Sun 16 May - Sat 5 June 2021

This program is tentative and subject to change.

Recently, there has been a significant growth of interest in applying software engineering techniques for the quality assurance of deep learning (DL) systems. One popular direction is deep learning testing, where adversarial examples (a.k.a.~bugs) of DL systems are found either by fuzzing or guided search with the help of certain testing metrics. However, recent studies have revealed that the commonly used neuron coverage metrics by existing DL testing approaches are not correlated to model robustness. It is also not an effective measurement on the confidence of the model robustness after testing. In this work, we address this gap by proposing a novel testing framework called \emph{Rob}ustness-\emph{O}riented \emph{T}esting (RobOT). A key part of RobOT is a quantitative measurement on 1) the value of each test case in improving model robustness (often via retraining), and 2) the convergence quality of the model robustness improvement. RobOT utilizes the proposed metrics to automatically generate test cases valuable for improving model robustness. The proposed metrics are also a strong indicator on how well robustness improvement has converged through testing. Experiments on multiple benchmark datasets confirm the effectiveness and efficiency of RobOT in improving DL model robustness, with 67.02% increase on the adversarial robustness that is 50.65% higher than the state-of-the-art work DeepGini.

This program is tentative and subject to change.

Wed 26 May
Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:20 - 12:20
2.1.2. Deep Neural Networks: Quality AssuranceTechnical Track at Blended Sessions Room 2
Chair(s): Gregorio RoblesUniversidad Rey Juan Carlos
11:20
20m
Paper
Graph-based Fuzz Testing for Deep Learning Inference EnginesTechnical Track
Technical Track
Weisi LuoI&V Dept of Kirin Solution Dept, HS, Huawei, Xiaoyue RunI&V Dept of Kirin Solution Dept, HS, Huawei, Dong ChaiI&V Dept of Kirin Solution Dept, HS, Huawei, Jiang WangI&V Dept of Kirin Solution Dept, HS, Huawei, Chunrong FangNanjing University, Zhenyu ChenNanjing University
Pre-print
11:40
20m
Paper
RobOT: Robustness-Oriented Testing for Deep Learning SystemsTechnical Track
Technical Track
Jingyi WangZhejiang University, Jialuo ChenZhejiang University, Youcheng SunQueen's University Belfast, UK, Xingjun MaDeakin University, Dongxia WangZhejiang University, Jun SunSingapore Management University, Singapore, Peng ChengZhejiang University
Pre-print
12:00
20m
Paper
Scalable Quantitative Verification For Deep Neural NetworksArtifact ReusableTechnical Track
Technical Track
Teodora BalutaNational University of Singapore, Zheng Leong ChuaIndependent Researcher, Kuldeep S. MeelNational University of Singapore, Prateek SaxenaNational University of Singapore
Pre-print