Empirical Evaluation of Frequency Based Statistical Models for Estimating Killable Mutants
Background. Mutation analysis is the premier technique to evaluate software test suite quality and estimating residual software defects. However, the reliability of mutation analysis is hampered by the so called equivalent mutants which are unkillable by any test case. Reliably detecting and eliminating killable mutants is difficult as it is highly program and location dependent. Hence, statistical estimation of killable mutants offers a promising approach.
Aims. Recently, frequency based species estimation methods has been proposed as a solution for related problems in software testing. While promising, there has been no comprehensive study on whether such frequency based species estimation methods can deliver an accurate estimate for killable mutants. Hence, this paper investigates the following: Can the frequency based estimation methods provide an accurate estimate for the number of killable mutants?
Method. In this paper, we report the results of a large-scale empirical study on the application of twelve widely known frequency based statistical models for estimating the number of killable mutants in ten mature software projects.
Result. Our investigation suggests that the considered statistical estimators lack sufficient predictive power and cannot produce reliable or useful estimates of killable mutants.
Thu 24 OctDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
16:00 - 17:30 | Software vulnerabilities and defectsESEM Technical Papers / ESEM Emerging Results, Vision and Reflection Papers Track / ESEM Journal-First Papers at Sala de graus (C4 Building) Chair(s): Daniela Cruzes Norwegian University of Science and Technology | ||
16:00 20mFull-paper | Automated Code-centric Software Vulnerability Assessment: How Far Are We? An Empirical Study in C/C++ ESEM Technical Papers Anh Nguyen The , Triet Le The University of Adelaide, Muhammad Ali Babar School of Computer Science, The University of Adelaide DOI Pre-print | ||
16:20 20mFull-paper | Empirical Evaluation of Frequency Based Statistical Models for Estimating Killable Mutants ESEM Technical Papers Konstantin Kuznetsov Saarland University, CISPA, Alessio Gambi Austrian Institute of Technology (AIT), Saikrishna Dhiddi Passau University, Julia Hess Saarland University, Rahul Gopinath University of Sydney | ||
16:40 20mFull-paper | Reevaluating the Defect Proneness of Atoms of Confusion in Java Systems ESEM Technical Papers Guoshuai Shi University of Waterloo, Farshad Kazemi University of Waterloo, Michael W. Godfrey University of Waterloo, Canada, Shane McIntosh University of Waterloo Pre-print | ||
17:00 15mVision and Emerging Results | DetectBERT: Towards Full App-Level Representation Learning to Detect Android Malware ESEM Emerging Results, Vision and Reflection Papers Track Tiezhu Sun University of Luxembourg, Nadia Daoudi Luxembourg Institute of Science and Technology, Kisub Kim Singapore Management University, Singapore, Kevin Allix Independent Researcher, Tegawendé F. Bissyandé University of Luxembourg, Jacques Klein University of Luxembourg | ||
17:15 15mJournal Early-Feedback | Identifying concerns when specifying machine learning-enabled systems: A perspective-based approach ESEM Journal-First Papers Hugo Villamizar fortiss GmbH, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Helio Côrtes Vieira Lopes PUC-Rio, Daniel Mendez Blekinge Institute of Technology and fortiss DOI | ||