ASDF - A Differential Testing Framework for Automatic Speech Recognition Systems
In this work, we propose an Automated Speech Recognition Differential Testing Framework (ASDF) for testing Automated Speech Recognition (ASR) systems. ASDF is an extension of CrossASR++, an existing ASR testing tool that automates the audio test case generation process. However, CrossASR++ does not make efficient use of the input corpus and does not provide useful information on the failed test cases for ASR improvement. To resolve this, ASDF fills in the gaps by implementing differential testing into CrossASR++ through various text transformation methods. ASDF provides a targeted and controlled increase in audio test case generation by applying text transformation methods to the original text inputs that failed during the ASR testing process, therefore effectively creating new high-quality test cases for ASR systems from the existing text corpus. ASDF eliminates the need for a large number of audio test cases to be sourced or generated for testing ASR systems, which is a weakness found in CrossASR++. In addition, ASDF records useful information for ASR analysis by using various metrics. ASDF also performs phonetic analysis on the failed test cases which identifies and graphs the phonemes that ASR systems tend to transcribe incorrectly, providing useful information for ASR developers and researchers in the future. Video demonstration: url{https://www.youtube.com/watch?v=DzVwfc3h9As}.
Mon 17 AprDisplayed time zone: Dublin change
11:00 - 12:30 | Session 1: Automated Testing Journal-First Papers / Research Papers / Previous Editions / Testing Tools / Tool Demo at Grand canal Chair(s): Gilles Perrouin Fonds de la Recherche Scientifique - FNRS & University of Namur | ||
11:00 20mTalk | JUGE: An infrastructure for benchmarking Java unit test generators Journal-First Papers Xavier Devroey University of Namur, Alessio Gambi IMC University of Applied Sciences Krems, Juan Pablo Galeotti University of Buenos Aires, René Just University of Washington, Fitsum Kifetew Fondazione Bruno Kessler, Annibale Panichella Delft University of Technology, Sebastiano Panichella Zurich University of Applied Sciences DOI Authorizer link Pre-print | ||
11:20 20mTalk | Metamorphic Testing with Causal Graphs Research Papers Andrew Graham Clark University of Sheffield, Michael Foster University of Sheffield, Neil Walkinshaw University of Sheffield, Robert Hierons University of Sheffield | ||
11:40 20mTalk | QEX: Automated Testing Observability and QA Developer Experience Framework Testing Tools | ||
12:00 10mTalk | ASDF - A Differential Testing Framework for Automatic Speech Recognition Systems Tool Demo Daniel Hao Xian Yuen School of Information Technology, Monash University Malaysia, Andrew Yong Chen Pang School of Information Technology, Monash University Malaysia, Zhou Yang Singapore Management University, Chun Yong Chong Monash University Malaysia, Mei Kuan Lim Monash University Malaysia, David Lo Singapore Management University | ||
12:10 20mTalk | A Framework for Automated API Fuzzing at Enterprise Scale Previous Editions Riyadh Mahmood The Aerospace Corporation, Jay Pennington The Aerospace Corporation, Danny Tsang The Aerospace Corporation, Tan Tran The Aerospace Corporation, Andrea Bogle The Aerospace Corporation DOI |