Enhancing the defectiveness prediction of methods and classes via JIT
Context
Defect prediction can help at prioritizing testing tasks by, for instance, ranking a list of items (methods and classes) according to their likelihood to be defective. While many studies investigated how to predict the defectiveness of commits, methods, or classes separately, no study investigated how these predictions differ or benefit each other. Specifically, at the end of a release, before the code is shipped to production, testing can be aided by ranking methods or classes, and we do not know which of the two approaches is more accurate. Moreover, every commit touches one or more methods in one or more classes; hence, the likelihood of a method and a class being defective can be associated with the likelihood of the touching commits being defective. Thus, it is reasonable to assume that the accuracy of methods-defectiveness-predictions (MDP) and the class-defectiveness-predictions (CDP) are increased by leveraging commits-defectiveness-predictions (aka JIT).
Objective
The contribution of this paper is fourfold: (i) We compare methods and classes in terms of defectiveness and (ii) of accuracy in defectiveness prediction, (iii) we propose and evaluate a first and simple approach that leverages JIT to increase MDP accuracy and (iv) CDP accuracy.
Method
We analyse accuracy using two types of metrics (threshold-independent and effort-aware). We also use feature selection metrics, nine machine learning defect prediction classifiers, more than 2.000 defects related to 38 releases of nine open source projects from the Apache ecosystem. Our results are based on a ground truth with a total of 285,139 data points and 46 features among commits, methods and classes.
Results
Our results show that leveraging JIT by using a simple median approach increases the accuracy of MDP by an average of 17% AUC and 46% PofB10 while it increases the accuracy of CDP by an average of 31% AUC and 38% PofB20.
Conclusions
From a practitioner’s perspective, it is better to predict and rank defective methods than defective classes. From a researcher’s perspective, there is a high potential for leveraging statement-defectiveness-prediction (SDP) to aid MDP and CDP.
Wed 13 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:30 - 12:00 | Code Quality and Code SmellsTool Demonstrations / Journal-first Papers / Research Papers at Plenary Room 2 Chair(s): Bernd Fischer Stellenbosch University | ||
10:30 12mTalk | Contextuality of Code Representation Learning Research Papers Yi Li New Jersey Institute of Technology, Shaohua Wang New Jersey Institute of Technology, Tien N. Nguyen University of Texas at Dallas | ||
10:42 12mTalk | On-the-fly Improving Performance of Deep Code Models via Input Denoising Research Papers Pre-print File Attached | ||
10:54 12mTalk | Using Deep Learning to Automatically Improve Code Readability Research Papers Antonio Vitale University of Molise, Italy, Valentina Piantadosi University of Molise, Simone Scalabrino University of Molise, Rocco Oliveto University of Molise Pre-print | ||
11:06 12mTalk | Towards Automatically Addressing Self-Admitted Technical Debt: How Far Are We? Research Papers Antonio Mastropaolo Università della Svizzera italiana, Massimiliano Di Penta University of Sannio, Italy, Gabriele Bavota Software Institute, USI Università della Svizzera italiana Pre-print File Attached | ||
11:18 12mTalk | How to Find Actionable Static Analysis Warnings: A Case Study with FindBugs Journal-first Papers Rahul Yedida , Hong Jin Kang UCLA, Huy Tu North Carolina State University, USA, Xueqi Yang NCSU, David Lo Singapore Management University, Tim Menzies North Carolina State University Link to publication DOI Authorizer link Pre-print | ||
11:30 12mTalk | Polyglot Code Smell Detection for Infrastructure as Code with GLITCH Tool Demonstrations Nuno Saavedra INESC-ID and IST, University of Lisbon, João Gonçalves INESC-ID and IST, University of Lisbon, Miguel Henriques INESC-ID and IST, University of Lisbon, João F. Ferreira INESC-ID and IST, University of Lisbon, Alexandra Mendes Faculty of Engineering, University of Porto & INESC TEC Pre-print File Attached | ||
11:42 12mTalk | Enhancing the defectiveness prediction of methods and classes via JIT Journal-first Papers Falessi Davide University of Rome Tor Vergata, Simone Mesiano Laureani University of Rome Tor Vergata, Jonida Çarka University of Rome Tor Vergata, Matteo Esposito University of Rome Tor Vergata, Daniel Alencar Da Costa University of Otago Link to publication DOI File Attached | ||