Quality Evaluation of Modern Code Reviews Through Intelligent Biometric Program Comprehension
Modern code reviews, typically associated with pull requests in version control systems such as Git, are a common practice in the software industry. However, individual reviewers may encounter challenges such as code comprehension difficulties, stress, fatigue, or distraction that can negatively impact the quality of their review task, particularly in what concerns the detection of possible bugs in the code under review. In this paper, we present a new technique for evaluating the quality of code reviews by assessing the reviewer’s cognitive load. Our technique utilizes biometric information collected from non-intrusive sensors and wearable measures, such as Heart Rate Variability (HRV) and task-evoked pupillary response, to estimate the cognitive load and code comprehension level of the reviewer. We also use an inexpensive desktop eye-tracker to identify code regions (i.e., snippets of code under review) that are not well-reviewed. To predict the quality of code review, we employ explainable machine learning models that consider both biometric and non-biometric features such as the reviewer’s expertise, code complexity, number of code region revisits, and review time. Our experimental results show that our approach can accurately predict review quality, with a prediction accuracy of 87.77% and a Spearman correlation coefficient of 0.85, indicating a strong correlation between the predicted and actual review performance. This evaluation also demonstrates the accuracy of our cognitive load measurement using EEG signals as a reference point for the HRV and pupil signals. The proposed technique has the potential to improve the quality of code review in the software industry by automatically identifying code regions that were not thoroughly reviewed and enabling reviewers to take corrective actions.
Thu 18 MayDisplayed time zone: Hobart change
11:00 - 12:30 | Code reviewJournal-First Papers / SEIP - Software Engineering in Practice / Technical Track at Meeting Room 103 Chair(s): Thomas LaToza George Mason University | ||
11:00 15mTalk | Workflow analysis of data science code in public GitHub repositories Journal-First Papers Dhivyabharathi Ramasamy Department of Informatics, University of Zurich, Zurich, Switzerland, Cristina Sarasua Department of Informatics, University of Zurich, Zurich, Switzerland, Alberto Bacchelli University of Zurich, Abraham Bernstein Department of Informatics, University of Zurich, Zurich, Switzerland | ||
11:15 15mTalk | Quality Evaluation of Modern Code Reviews Through Intelligent Biometric Program Comprehension Journal-First Papers Haytham Hijazi CISUC, DEI, University of Coimbra, João Durães CISUC, Polytechnic Institute of Coimbra, Ricardo Couceiro University of Coimbra, Raul Barbosa CISUC, DEI, University of Coimbra, João Castelhano ICNAS, University of Coimbra, Júlio Medeiros CISUC, DEI, University of Coimbra, Miguel Castelo Branco ICNAS/CIBIT, University of Coimbra, Paulo Carvalho University of Coimbra, Henrique Madeira University of Coimbra | ||
11:30 15mTalk | Code Review of Build System Specifications: Prevalence, Purposes, Patterns, and Perceptions Technical Track Mahtab Nejati University of Waterloo, Mahmoud Alfadel University of Waterloo, Shane McIntosh University of Waterloo Pre-print | ||
11:45 15mTalk | Please fix this mutant: How do developers resolve mutants surfaced during code review? SEIP - Software Engineering in Practice Goran Petrovic Google; Universität Passau, René Just University of Washington, Marko Ivanković Google; Universität Passau, Gordon Fraser University of Passau | ||
12:00 15mTalk | Using Large-scale Heterogeneous Graph Representation Learning for Code Review Recommendations at Microsoft SEIP - Software Engineering in Practice Jiyang Zhang University of Texas at Austin, Chandra Maddila Microsoft Research, Ramakrishna Bairi Microsoft Research, Christian Bird Microsoft Research, Ujjwal Raizada Microsoft Research, Apoorva Agrawal Microsoft Research, Yamini Jhawar Microsoft Research, Kim Herzig Microsoft, Arie van Deursen Delft University of Technology Pre-print Media Attached | ||
12:15 7mTalk | A mixed-methods analysis of micro-collaborative coding practices in OpenStack Journal-First Papers Armstrong Foundjem Queen's University, Eleni Constantinou University of Cyprus, Tom Mens University of Mons, Bram Adams Queen's University, Kingston, Ontario |