ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland
Mon 17 Apr 2023 12:00 - 12:10 at Grand canal - Session 1: Automated Testing Chair(s): Gilles Perrouin

In this work, we propose an Automated Speech Recognition Differential Testing Framework (ASDF) for testing Automated Speech Recognition (ASR) systems. ASDF is an extension of CrossASR++, an existing ASR testing tool that automates the audio test case generation process. However, CrossASR++ does not make efficient use of the input corpus and does not provide useful information on the failed test cases for ASR improvement. To resolve this, ASDF fills in the gaps by implementing differential testing into CrossASR++ through various text transformation methods. ASDF provides a targeted and controlled increase in audio test case generation by applying text transformation methods to the original text inputs that failed during the ASR testing process, therefore effectively creating new high-quality test cases for ASR systems from the existing text corpus. ASDF eliminates the need for a large number of audio test cases to be sourced or generated for testing ASR systems, which is a weakness found in CrossASR++. In addition, ASDF records useful information for ASR analysis by using various metrics. ASDF also performs phonetic analysis on the failed test cases which identifies and graphs the phonemes that ASR systems tend to transcribe incorrectly, providing useful information for ASR developers and researchers in the future. Video demonstration: url{https://www.youtube.com/watch?v=DzVwfc3h9As}.

Mon 17 Apr

Displayed time zone: Dublin change

11:00 - 12:30
Session 1: Automated Testing Journal-First Papers / Research Papers / Previous Editions / Testing Tools / Tool Demo at Grand canal
Chair(s): Gilles Perrouin Fonds de la Recherche Scientifique - FNRS & University of Namur
JUGE: An infrastructure for benchmarking Java unit test generators
Journal-First Papers
Xavier Devroey University of Namur, Alessio Gambi IMC University of Applied Sciences Krems, Juan Pablo Galeotti University of Buenos Aires, René Just University of Washington, Fitsum Kifetew Fondazione Bruno Kessler, Annibale Panichella Delft University of Technology, Sebastiano Panichella Zurich University of Applied Sciences
DOI Authorizer link Pre-print
Metamorphic Testing with Causal Graphs
Research Papers
Andrew Graham Clark University of Sheffield, Michael Foster University of Sheffield, Neil Walkinshaw University of Sheffield, Robert Hierons University of Sheffield
QEX: Automated Testing Observability and QA Developer Experience Framework
Testing Tools
Luohua Huang Shopee, Joseph Chu Shopee, Keshia Yap Shopee, Hock Yao Chua Shopee
ASDF - A Differential Testing Framework for Automatic Speech Recognition Systems
Tool Demo
Daniel Hao Xian Yuen School of Information Technology, Monash University Malaysia, Andrew Yong Chen Pang School of Information Technology, Monash University Malaysia, Zhou Yang Singapore Management University, Chun Yong Chong Monash University Malaysia, Mei Kuan Lim Monash University Malaysia, David Lo Singapore Management University
A Framework for Automated API Fuzzing at Enterprise Scale
Previous Editions
Riyadh Mahmood The Aerospace Corporation, Jay Pennington The Aerospace Corporation, Danny Tsang The Aerospace Corporation, Tan Tran The Aerospace Corporation, Andrea Bogle The Aerospace Corporation