The Pull-Based development model, a fundamental mechanism of collaboration in modern software engineering, initiates the code review process when a contributor submits pull requests (PRs) for evaluation. Although the decision to approve or decline PRs is often perceived as grounded in their technical quality, prior research presents a more intricate narrative where both social signals and technical factors influence the acceptance.

This study, uniquely integrating cues of urgency (represented by code priority level) and reputation (represented by the experience level of the code’s author), delves into these biases, leveraging eye-tracking technology to illuminate the cognitive processes underpinning the evaluation of PRs.

In an experimentally-controlled study involving 37 participants reviewing Java code patches, we found that perceived PR priority impacted both the time spent on tasks and the associated cognitive load. Moreover, while participants’ behaviors reflected the influence of urgency and reputation, they remained largely unaware of these effects on their decision-making, highlighting the critical importance of understanding these implicit biases in code reviews. Interestingly, despite variations in attention when reviewing contributions from novice versus senior authors, there was no discernible difference in acceptance outcomes based on the author’s experience. This study takes the next step toward a better understanding of urgency and reputation in software engineering and may inform future research about code review platforms and guidelines, code reuse, and automated code generation.