Write a Blog >>
ICSE 2022
Sun 8 - Fri 27 May 2022
Mon 9 May 2022 09:00 - 09:30 at MET room - Metamorphic Testing in Deep Learning Chair(s): Xiaoyuan Xie

Speech recognition (SR) systems are systems that convert speech signals into text and are widely used in mobile devices, wearable devices, and intelligent living room devices. When SR software recognizes speech in the laboratory or another quiet environment, the recognition accuracy is high. However, when SR software is applied in a complex real-world environment, there may be background noise from birds, machines, and so on, the influence of other speakers, or other adverse factors, which will reduce the recognition accuracy of the SR software. In this work, we propose SR-MT, which is a metamorphic testing (MT) approach to test the robustness of SR software. In SR-MT, we adopt four criteria to analyze the MT results and rank the robustness of the target SR software for speech affected by noise interference and speech variations from level 1 to level 5. SR-MT was evaluated on three real industrial applications: iFLYTEK speech-to-text1, Baidu speech-to-text2, and Google speech-to-text3. We found that on average, 13.5% of the words in the speech could not be recognized correctly when the signal-to-noise ratio reached 10 dB. Similarly, changes in speech speed and tone will also reduce the recognition accuracy of SR software.

Mon 9 May

Displayed time zone: Eastern Time (US & Canada) change

08:00 - 09:30
Metamorphic Testing in Deep LearningMET at MET room
Chair(s): Xiaoyuan Xie School of Computer Science, Wuhan University, China
08:00
30m
Talk
In-Place Metamorphic Exploration and Testing
MET
Zhi Quan (George) Zhou University of Wollongong, Australia, Junting Zhu University of Wollongong, Tsong Yueh Chen Swinburne University of Technology, Dave Towey University of Nottingham Ningbo China
08:30
30m
Talk
Fairness Evaluation in Deepfake Detection Models using Metamorphic Testing
MET
Muxin Pu Monash University Malaysia, Meng Yi Kuan Monash University Malaysia, Nyee Thoang Lim Monash University Malaysia, Mei Kuan Lim Monash University Malaysia, Chun Yong Chong Monash University Malaysia
09:00
30m
Talk
SR-MT:A Metamorphic Method to Test Robustness of Speech Recognition Software
MET
Feifei Wang Naval University of Engineering, Kerong Ben Naval University of Engineering, Xian Zhang Naval University of Engineering

Information for Participants
Mon 9 May 2022 08:00 - 09:30 at MET room - Metamorphic Testing in Deep Learning Chair(s): Xiaoyuan Xie
Info for room MET room:

Click here to go to the room on Midspace