Write a Blog >>
ICSE 2022
Sun 8 - Fri 27 May 2022
Mon 9 May 2022 08:00 - 08:30 at MET room - Metamorphic Testing in Deep Learning Chair(s): Xiaoyuan Xie

In this study, we consider the testing of software whose input involves a data stream, such as in video analysis applications. To address the oracle problem in testing this kind of software, existing metamorphic testing (MT) and metamorphic exploration (ME) techniques typically generate both source and follow-up input streams. In contrast, we propose a different strategy: an in-place method. Instead of examining relations among multiple input streams, our method works on a finer granularity by investigating relations among the data frames within a single input stream — such as the relations among the image frames within a video stream — treating a frame, rather than a stream, as a test case. Because our method does not generate any follow-up input stream, or change any frame values, it is also suitable for runtime error detection. We have applied this in-place method to testing video streams in Baidu Apollo, a real-life autonomous driving system. Our study identified previously-unknown obstacle perception failures in the camera perception module, including both undetected and incorrectly-detected objects. The empirical results show that our approach is practical, and readily applicable to industrial-scale systems that include (but are not limited to) the computer vision domain.

Mon 9 May

Displayed time zone: Eastern Time (US & Canada) change

08:00 - 09:30
Metamorphic Testing in Deep LearningMET at MET room
Chair(s): Xiaoyuan Xie School of Computer Science, Wuhan University, China
08:00
30m
Talk
In-Place Metamorphic Exploration and Testing
MET
Zhi Quan (George) Zhou University of Wollongong, Australia, Junting Zhu University of Wollongong, Tsong Yueh Chen Swinburne University of Technology, Dave Towey University of Nottingham Ningbo China
08:30
30m
Talk
Fairness Evaluation in Deepfake Detection Models using Metamorphic Testing
MET
Muxin Pu Monash University Malaysia, Meng Yi Kuan Monash University Malaysia, Nyee Thoang Lim Monash University Malaysia, Mei Kuan Lim Monash University Malaysia, Chun Yong Chong Monash University Malaysia
09:00
30m
Talk
SR-MT:A Metamorphic Method to Test Robustness of Speech Recognition Software
MET
Feifei Wang Naval University of Engineering, Kerong Ben Naval University of Engineering, Xian Zhang Naval University of Engineering

Information for Participants
Mon 9 May 2022 08:00 - 09:30 at MET room - Metamorphic Testing in Deep Learning Chair(s): Xiaoyuan Xie
Info for room MET room:

Click here to go to the room on Midspace