Modern code review is central to software engineering processes, and many undergraduate software engineering courses teach it in some form. To unveil barriers students face when evaluating code changes during review, we combine interviews with an observational study and survey. In a junior-level software engineering course, we first interviewed 29 undergraduate students about their experiences in code review. Next, we performed an observational study that presented 44 students from the same course with eight code change comprehension activities. These activities provided students with pull requests of potential refactorings in a familiar code base, collecting feedback on accuracy and challenges. This was followed by a reflection survey.

We combine (1) a qualitative analysis of the interview transcripts, activity comments, and reflection survey with (2) a quantitative assessment of their performance in identifying behavioral changes to identify barriers students face during code change comprehension. Our findings reveal that students experience barriers related to code review context, code review tools, the code itself, and comprehending code changes. These findings–along with our result that student developers tend to overestimate behavioral similarity during code comparison–have implications for future support to help student developers have smoother code review experiences. We motivate a need for several interventions, including but not limited to sentiment analysis on pull request comments to flag toxicity, scaffolding for code comprehension while reviewing large changes, and behavioral diffing to show how variable values change alongside a proposed code change.