How Do Developers Use Code Suggestions in Pull Request Reviews?
Background: GitHub introduced the suggestion feature to enable reviewers to explicitly suggest code modifications in pull requests. These suggestions make the reviewers’ feedback more actionable for the submitters and can act as a valuable knowledge source for newcomers. Still, little is known about how code review suggestions are used by developers, what impact they have on pull requests and how they are influenced by social coding dynamics. Aims: To bridge this knowledge gap, we conducted an empirical study on pull requests from 46 engineered GitHub projects, in which developers used code review suggestions. Method: We applied an open coding approach to uncover the types of suggestions and their usage frequency. We also mined pull request characteristics and assessed the impact of using suggestions on merge rate and resolution time as well as code complexity. Furthermore, we conducted a survey with the contributors of the projects we studied to gain insights about the influence of social factors on the usage and acceptance of code review suggestions. Results: We were able to uncover four suggestion types: code style suggestions, improvements, fixes, and documentation with improvements being the most frequent. We found that the use of suggestions positively affects the merge rate of pull requests but significantly increases resolution time without leading to a decrease in code complexity. Our survey results show that suggestions are more likely to be used by reviewers when the submitter is a newcomer. The results also show that developers mostly search suggestions when tracking rationale and when looking for code examples. Conclusion: Our work provides insights on the usage and impact of code review suggestions and provides recommendations for future automation efforts.