Write a Blog >>
ICSE 2021
Mon 17 May - Sat 5 June 2021

This program is tentative and subject to change.

With the wide use of Deep Learning (DL) systems, academy and industry begin to pay attention to their quality. Testing is one of the major methods of quality assurance. However, existing testing techniques focus on the quality of DL models but lacks attention to the core underlying inference engines (i.e., frameworks and libraries). Inspired by the success stories of fuzz testing, we design a graph-based fuzz testing method to improve the quality of DL inference engines. This method is naturally followed by the graph structure of DL models. An operator-level coverage based on graph theory is introduced and six different mutations are implemented to generate diversified DL models by exploring combinations of model structures, parameters, and data. The Monte Carlo Tree Search (MCTS) is used to drive DL model generation without a training process. The experimental results show that the MCTS outperforms the random method in boosting operator-level coverage and detecting exceptions. Our method has discovered more than 40 different exceptions in three types of undesired behaviors: model conversion failure, inference failure, output comparison failure. The mutation strategies are useful to generate new valid test inputs, by up to an 8.2% more operator-level coverage on average and 8.6 more exceptions captured.

Graph-based fuzz testing for deep learning inference engines (video) (presentation_compressed.mp4)18.14MiB

This program is tentative and subject to change.

Wed 26 May
Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:20 - 12:20
2.1.2. Deep Neural Networks: Quality AssuranceTechnical Track at Blended Sessions Room 2 +12h
Chair(s): Gregorio RoblesUniversidad Rey Juan Carlos
11:20
20m
Paper
Graph-based Fuzz Testing for Deep Learning Inference EnginesTechnical Track
Technical Track
Weisi LuoI&V Dept of Kirin Solution Dept, HS, Huawei, Xiaoyue RunI&V Dept of Kirin Solution Dept, HS, Huawei, Dong ChaiI&V Dept of Kirin Solution Dept, HS, Huawei, Jiang WangI&V Dept of Kirin Solution Dept, HS, Huawei, Chunrong FangNanjing University, Zhenyu ChenNanjing University
Pre-print File Attached
11:40
20m
Paper
RobOT: Robustness-Oriented Testing for Deep Learning SystemsTechnical Track
Technical Track
Jingyi WangZhejiang University, Jialuo ChenZhejiang University, Youcheng SunQueen's University Belfast, UK, Xingjun MaDeakin University, Dongxia WangZhejiang University, Jun SunSingapore Management University, Singapore, Peng ChengZhejiang University
Pre-print
12:00
20m
Paper
Scalable Quantitative Verification For Deep Neural NetworksArtifact ReusableTechnical Track
Technical Track
Teodora BalutaNational University of Singapore, Zheng Leong ChuaIndependent Researcher, Kuldeep S. MeelNational University of Singapore, Prateek SaxenaNational University of Singapore
Pre-print
23:20 - 00:20
2.1.2. Deep Neural Networks: Quality AssuranceTechnical Track at Blended Sessions Room 2
23:20
20m
Paper
Graph-based Fuzz Testing for Deep Learning Inference EnginesTechnical Track
Technical Track
Weisi LuoI&V Dept of Kirin Solution Dept, HS, Huawei, Xiaoyue RunI&V Dept of Kirin Solution Dept, HS, Huawei, Dong ChaiI&V Dept of Kirin Solution Dept, HS, Huawei, Jiang WangI&V Dept of Kirin Solution Dept, HS, Huawei, Chunrong FangNanjing University, Zhenyu ChenNanjing University
Pre-print File Attached
23:40
20m
Paper
RobOT: Robustness-Oriented Testing for Deep Learning SystemsTechnical Track
Technical Track
Jingyi WangZhejiang University, Jialuo ChenZhejiang University, Youcheng SunQueen's University Belfast, UK, Xingjun MaDeakin University, Dongxia WangZhejiang University, Jun SunSingapore Management University, Singapore, Peng ChengZhejiang University
Pre-print
00:00
20m
Paper
Scalable Quantitative Verification For Deep Neural NetworksArtifact ReusableTechnical Track
Technical Track
Teodora BalutaNational University of Singapore, Zheng Leong ChuaIndependent Researcher, Kuldeep S. MeelNational University of Singapore, Prateek SaxenaNational University of Singapore
Pre-print

Information for Participants
Info for Blended Sessions Room 2: