Good Things Come In Threes: Improving Search-based Crash Reproduction With Helper Objectives
Evolutionary intelligence approaches have been successfully applied to assist developers during debugging by generating a test case reproducing reported crashes. These approaches use a single fitness function called Crash Distance to guide the search process toward reproducing a target crash. Despite the reported achievements, these approaches do not always successfully reproduce some crashes due to a lack of test diversity (premature convergence). In this study, we introduce a new approach, called MO-HO, that addresses this issue via multi-objectivization. In particular, we introduce two new Helper-Objectives for crash reproduction, namely test length (to minimize) and method sequence diversity (to maximize), in addition to Crash Distance. We assessed MO-HO using five multi-objective evolutionary algorithms (NSGA-II, SPEA2, PESA-II, MOEA/D, FEMO) on 124 hard-to-reproduce crashes stemming from open-source projects. Our results indicate that SPEA2 is the best-performing multi-objective algorithm for MO-HO. We evaluated this best-performing algorithm for MO-HO against the state-of-the-art: single-objective approach (Single-Objective Search) and decomposition-based multi-objectivization approach (De-MO). Our results show that MO-HO reproduces five crashes that cannot be reproduced by the current state-of-the-art. Besides, MO-HO improves the effectiveness (+10% and +8% in reproduction ratio) and the efficiency in 34.6% and 36% of crashes (i.e., significantly lower running time) compared to Single-Objective Search and De-MO, respectively. For some crashes, the improvements are very large, being up to +93.3% for reproduction ratio and -92% for the required running time.
Tue 22 SepDisplayed time zone: (UTC) Coordinated Universal Time change
09:10 - 10:10 | Search-Based TestingJournal-first Papers / Tool Demonstrations / Research Papers at Wombat Chair(s): Maria Kechagia University College London | ||
09:10 20mTalk | Good Things Come In Threes: Improving Search-based Crash Reproduction With Helper Objectives Research Papers Pouria Derakhshanfar Delft University of Technology, Xavier Devroey Delft University of Technology, Andy Zaidman Delft University of Technology, Arie van Deursen Delft University of Technology, Netherlands, Annibale Panichella Delft University of Technology DOI Pre-print Media Attached | ||
09:30 20mTalk | Multi-criteria test cases selection for model transformations Journal-first Papers Bader Alkhazi Kuwait University, Chaima Abid University of Michigan, Marouane Kessentini University of Michigan, Dorian Leroy JKU Linz, Manuel Wimmer Johannes Kepler University Linz Link to publication DOI | ||
09:50 10mTalk | Botsing, a Search-based Crash Reproduction Framework for Java Tool Demonstrations Pouria Derakhshanfar Delft University of Technology, Xavier Devroey Delft University of Technology, Annibale Panichella Delft University of Technology, Andy Zaidman Delft University of Technology, Arie van Deursen Delft University of Technology, Netherlands DOI Pre-print Media Attached |