ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland
Mon 17 Apr 2023 11:20 - 11:40 at Grand canal - Session 1: Automated Testing Chair(s): Gilles Perrouin

Metamorphic testing provides a means by which to generate succinct test oracles that can apply to large input spaces. For this it depends on the formulation of metamorphic relations, which generally require extensive domain expertise and human input. To address this problem, we present a model-based testing approach that can automatically generate metamorphic relations and associated tests. Our approach is motivated by the observation that metamorphic testing is a fundamentally causal task. We show how it is possible to leverage lightweight graph-based modelling techniques from the field of Causal Inference to specify causal properties of the system-under-test. Through a series of controlled experiments, we find that the proposed approach is robust to misspecification and can test evasive causal relationships (i.e. those that are difficult to exercise and observe) when combined with an appropriate test generation strategy. We also apply the approach to two case studies from the Defects4J framework with known bugs that affect causal behaviour. The results of these case studies suggest that the approach is not only useful for catching bugs affecting causal structure, but also alerting the user to inaccuracies in the specification.

Mon 17 Apr

Displayed time zone: Dublin change

11:00 - 12:30
Session 1: Automated Testing Journal-First Papers / Research Papers / Previous Editions / Testing Tools / Tool Demo at Grand canal
Chair(s): Gilles Perrouin Fonds de la Recherche Scientifique - FNRS & University of Namur
11:00
20m
Talk
JUGE: An infrastructure for benchmarking Java unit test generators
Journal-First Papers
Xavier Devroey University of Namur, Alessio Gambi IMC University of Applied Sciences Krems, Juan Pablo Galeotti University of Buenos Aires, René Just University of Washington, Fitsum Kifetew Fondazione Bruno Kessler, Annibale Panichella Delft University of Technology, Sebastiano Panichella Zurich University of Applied Sciences
DOI Authorizer link Pre-print
11:20
20m
Talk
Metamorphic Testing with Causal Graphs
Research Papers
Andrew Graham Clark University of Sheffield, Michael Foster University of Sheffield, Neil Walkinshaw University of Sheffield, Robert Hierons University of Sheffield
11:40
20m
Talk
QEX: Automated Testing Observability and QA Developer Experience Framework
Testing Tools
Luohua Huang Shopee, Joseph Chu Shopee, Keshia Yap Shopee, Hock Yao Chua Shopee
12:00
10m
Talk
ASDF - A Differential Testing Framework for Automatic Speech Recognition Systems
Tool Demo
Daniel Hao Xian Yuen School of Information Technology, Monash University Malaysia, Andrew Yong Chen Pang School of Information Technology, Monash University Malaysia, Zhou Yang Singapore Management University, Chun Yong Chong Monash University Malaysia, Mei Kuan Lim Monash University Malaysia, David Lo Singapore Management University
12:10
20m
Talk
A Framework for Automated API Fuzzing at Enterprise Scale
Previous Editions
Riyadh Mahmood The Aerospace Corporation, Jay Pennington The Aerospace Corporation, Danny Tsang The Aerospace Corporation, Tan Tran The Aerospace Corporation, Andrea Bogle The Aerospace Corporation
DOI