Write a Blog >>
ISSTA 2021
Sun 11 - Sat 17 July 2021 Online
co-located with ECOOP and ISSTA 2021

Deep learning(DL) techniques attract people from various fields with superior performance in making progressive breakthroughs. To ensure the quality of DL techniques, researchers have been working on testing and verification approaches. Some recent studies reveal that the underlying DL operators could cause defects inside a DL model. DL operators work as fundamental components in DL libraries. Library developers still work on practical approaches to ensure the quality of operators they provide. However, the variety of DL operators and the implementation complexity make it challenging to evaluate their quality. Operator testing with limited test cases may fail to reveal hidden defects inside the implementation. Besides, the existing model-to-library testing approach requires extra labor and time cost to identify and locate errors, i.e., developers can only react to the exposed defects. This paper proposes a fuzzing-based operator-level precision testing approach to estimate individual DL operators' precision errors to bridge this gap. Unlike conventional fuzzing techniques, valid shape variable inputs and fine-grained precision error evaluation are implemented. The testing of DL operators is treated as a searching problem to maximize output precision errors. We implement our approach in a tool named Predoo and conduct an experiment on seven DL operators from TensorFlow. The experiment result shows that Predoo can trigger larger precision errors compared to the error threshold declared in the testing scripts from the TensorFlow repository.

Thu 15 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

19:00 - 20:20
Session 11 (time band 1) Machine Learning and TestingTechnical Papers at ISSTA 1
Chair(s): August Shi University of Texas at Austin
19:00
20m
Talk
Interval Constraint-Based Mutation Testing of Numerical Specifications
Technical Papers
Clothilde Jeangoudoux MPI-SWS, Eva Darulova MPI-SWS, Christoph Lauter University of Alaska at Anchorage
DOI
19:20
20m
Talk
Predoo: Precision Testing of Deep Learning Operators
Technical Papers
Xufan Zhang Nanjing University, Ning Sun Nanjing University, Chunrong Fang Nanjing University, Jiawei Liu Nanjing University, Jia Liu Nanjing University, Dong Chai Huawei, Jiang Wang Huawei, Zhenyu Chen Nanjing University
DOI
19:40
20m
Talk
TERA: Optimizing Stochastic Regression Tests in Machine Learning Projects
Technical Papers
Saikat Dutta University of Illinois at Urbana-Champaign, Jeeva Selvam University of Illinois at Urbana-Champaign, Aryaman Jain University of Illinois at Urbana-Champaign, Sasa Misailovic University of Illinois at Urbana-Champaign
DOI
20:00
20m
Talk
Deep Just-in-Time Defect Prediction: How Far Are We?
Technical Papers
Zhengran Zeng Southern University of Science and Technology, Yuqun Zhang Southern University of Science and Technology, Haotian Zhang Kwai, Lingming Zhang University of Illinois at Urbana-Champaign
DOI

Fri 16 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

10:00 - 11:20
Session 17 (time band 3) Testing Miscellaneous Systems Technical Papers at ISSTA 1
Chair(s): Thuan Pham The University of Melbourne
10:00
20m
Talk
Empirical Evaluation of Smart Contract Testing: What Is the Best Choice?
Technical Papers
Meng Ren Tsinghua University, Zijing Yin Tsinghua University, Fuchen Ma Tsinghua University, Zhenyang Xu University of Waterloo, Yu Jiang Tsinghua University, Chengnian Sun University of Waterloo, Huizhong Li WeBank, Yan Cai Institute of Software at Chinese Academy of Sciences
DOI File Attached
10:20
20m
Talk
Interval Constraint-Based Mutation Testing of Numerical Specifications
Technical Papers
Clothilde Jeangoudoux MPI-SWS, Eva Darulova MPI-SWS, Christoph Lauter University of Alaska at Anchorage
DOI
10:40
20m
Talk
TERA: Optimizing Stochastic Regression Tests in Machine Learning Projects
Technical Papers
Saikat Dutta University of Illinois at Urbana-Champaign, Jeeva Selvam University of Illinois at Urbana-Champaign, Aryaman Jain University of Illinois at Urbana-Champaign, Sasa Misailovic University of Illinois at Urbana-Champaign
DOI
11:00
20m
Talk
Predoo: Precision Testing of Deep Learning Operators
Technical Papers
Xufan Zhang Nanjing University, Ning Sun Nanjing University, Chunrong Fang Nanjing University, Jiawei Liu Nanjing University, Jia Liu Nanjing University, Dong Chai Huawei, Jiang Wang Huawei, Zhenyu Chen Nanjing University
DOI