Write a Blog >>
ICSE 2021
Mon 17 May - Sat 5 June 2021

Code review is a broadly adopted software quality practice where developers critique each others’ patches. In addition to providing constructive feedback, reviewers may provide a score to indicate whether the patch should be integrated. Since reviewer opinions may differ, patches can receive both positive and negative scores. If reviews with divergent scores are not carefully resolved, they may contribute to a tense reviewing culture and may slow down integration. In this paper, we study patches with divergent review scores in the OPENSTACK and QT communities. Quantitative analysis indicates that patches with divergent review scores: (1) account for 15%–37% of patches that receive multiple review scores; (2) are integrated more often than they are abandoned; and (3) receive negative scores after positive ones in 70% of cases. Furthermore, a qualitative analysis indicates that patches with strongly divergent scores that: (4) are abandoned more often suffer from external issues (e.g., integration planning, content duplication) than patches with weakly divergent scores and patches without divergent scores; and (5) are integrated often address reviewer concerns indirectly (i.e., without changing patches). Our results suggest that review tooling should integrate with release schedules and detect concurrent development of similar patches to optimize review discussions with divergent scores. Moreover, patch authors should note that even the most divisive patches are often integrated through discussion, integration timing, and careful revision.

Conference Day
Wed 26 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:20 - 12:20
2.1.5. Code Review: Observational StudiesJournal-First Papers at Blended Sessions Room 5 +12h
Chair(s): Sira VegasUniversidad Politecnica de Madrid
11:20
20m
Paper
Review Dynamics and Their Impact on Software QualityJournal-First
Journal-First Papers
Patanamon ThongtanunamUniversity of Melbourne, Ahmed E. HassanSchool of Computing, Queen's University
DOI Pre-print Media Attached
11:40
20m
Paper
Code Reviews with Divergent Review Scores: An Empirical Study of the OpenStack and Qt CommunitiesJournal-First
Journal-First Papers
Toshiki HiraodTosh Inc.,, Shane McIntosh, Akinori IharaWakayama University, Kenichi MatsumotoNara Institute of Science and Technology
DOI Pre-print Media Attached
12:00
20m
Paper
A Longitudinal Study of Static Analysis Warning Evolution and the Effects of PMD on Software Quality in Apache Open Source ProjectsJournal-First
Journal-First Papers
Alexander TrautschUniversity of Göttingen, Steffen HerboldUniversity of Göttingen, Jens GrabowskiUniversity of Göttingen
Link to publication DOI Pre-print Media Attached