How Useful is Code Change Information for Fault Localization in Continuous Integration?
Continuous integration (CI) is a software practice by which developers frequently merge and test code under development. In CI settings, the change information is finer-grained. Prior studies have widely studied and evaluated the performance of spectrum-based fault localization (SBFL) techniques. While the continuous nature of CI requires the code changes to be atomic and presents fine-grained information on what part of the system is being changed, traditional SBFL techniques do not benefit from it. In this paper, we conduct an empirical study on the effectiveness of using and integrating code and coverage changes for fault localization in CI settings. We conduct our study on seven open source systems, with a total of 192 faults. We find that while both change information covers a reduced search space compared to code coverage, the percentages of faulty methods in the search space are 7 and 14 times higher than code coverage for code changes and coverage changes, respectively. Then, we propose three change-based fault localization techniques and compare them with Ochiai, a commonly used SBFL technique. Our results show that all three change-based techniques outperform Ochiai, achieving an improvement that varies from 7% to 23% and 17% to 24% over Ochiai for average MAP and MRR, respectively. Moreover, we find that our change-based fault localization techniques can be integrated with Ochiai, achieving up to 53% and 52% improvement over Ochiai in average MAP and MRR respectively, and locating 41 more faults at Top-1.
Wed 12 OctDisplayed time zone: Eastern Time (US & Canada) change
13:30 - 15:30 | Technical Session 14 - Bug Prediction and LocalizationJournal-first Papers / Research Papers / NIER Track / Industry Showcase at Banquet A Chair(s): David Lo Singapore Management University | ||
13:30 20mResearch paper | How Useful is Code Change Information for Fault Localization in Continuous Integration? Research Papers An Ran Chen Concordia University, Tse-Hsun (Peter) Chen Concordia University, Junjie Chen Tianjin University | ||
13:50 20mIndustry talk | Code Understanding Linter to Detect Variable Misuse Industry Showcase Yeonhee Ryou Samsung Research, Samsung Electronics, Sangwoo Joh Samsung Research, Samsung Electronics, Joonmo Yang Samsung Research, Samsung Electronics, Sujin Kim Samsung Research, Samsung Electronics, Youil Kim Samsung Research, Samsung Electronics | ||
14:10 20mPaper | Static Data-Flow Analysis for Software Product Lines in C Journal-first Papers Philipp Dominik Schubert Heinz Nixdorf Institut, Paderborn University, Paul Gazzillo University of Central Florida, Zachary Patterson University of Texas at Dallas, Julian Braha University of Central Florida, Fabian Schiebel Fraunhofer IEM, Ben Hermann Technical University Dortmund, Shiyi Wei University of Texas at Dallas, Eric Bodden University of Paderborn; Fraunhofer IEM DOI | ||
14:30 10mVision and Emerging Results | Boosting Spectrum-Based Fault Localization for Spreadsheets with Product Metrics in a Learning ApproachVirtual NIER Track Adil mukhtar Graz University of Technology, Birgit Hofer Graz University of Technology, Dietmar Jannach University of Klagenfurt, Franz Wotawa Graz University of Technology, Konstantin Schekotihin University of Klagenfurt | ||
14:40 20mResearch paper | Evolving Ranking-Based Failure Proximities for Better Clustering in Fault IsolationVirtual Research Papers Yi Song School of Computer Science, Wuhan University, Xiaoyuan Xie School of Computer Science, Wuhan University, China, Xihao Zhang School of Computer Science, Wuhan University, Quanming Liu School of Computer Science, Wuhan University, Ruizhi Gao Sonos Inc. | ||
15:00 20mPaper | Leveraging structural properties of source code graphs for just-in-time bug predictionVirtual Journal-first Papers Md Nadim University of Saskatchewan, Debajyoti Mondal University of Saskatchewan, Chanchal K. Roy University of Saskatchewan |