FSE 2026
Sun 5 - Thu 9 July 2026 Montreal, Canada
Wed 8 Jul 2026 10:50 - 11:10 at MB 2.210 - Code Review 1 Chair(s): Tao Xiao

Code review is central to collaborative software development, yet feedback quality can vary widely, influencing code maintainability and developer interactions. This study investigates how large language models (LLMs) can assess code review feedback quality along two dimensions, sentiment (with a focus on harmful comments) and specificity, to support more constructive collaboration. Using over 204,000 feedback threads from 30 open-source software (OSS) repositories, we evaluate eleven LLMs, achieving F1-scores up to 0.83 for sentiment and 0.67 for specificity. Most OSS feedback is neutral or low in specificity, with highly detailed or overtly harmful comments comprising a small minority. Industry data from 45 organisations contains significantly more highly specific feedback and more minimal reviews, while harmful feedback remains rare. Deployment of our approach in commercial settings demonstrated practical value. Specificity classifications delivered immediate value, such as revealing mentorship gaps when senior developers provided more specific feedback than they received, while harmful comment classifications required careful UX framing to avoid user sensitivity. Our findings demonstrate the feasibility and practical utility of automated feedback-quality assessment in real-world environments.

Wed 8 Jul

Displayed time zone: Eastern Time (US & Canada) change

10:30 - 12:30
Code Review 1Industry Papers / Journal-First Paper / Research Papers at MB 2.210
Chair(s): Tao Xiao Kyushu University
10:30
20m
Talk
SWR-Bench: Assessing LLM Performance in Real-World Code Review Comment Generation
Research Papers
Zhengran Zeng Peking University, Ruikai Shi Peking University, Keke Han Peking University, Yixin Li Peking University, Kaicheng Sun Northwestern Polytechnical University, Yidong Wang Peking University, Zhuohao Yu Peking University, Rui Xie Peking University, Wei Ye Peking University, Shikun Zhang Peking University
10:50
20m
Talk
Assessing Harmful Comments and Specificity in Code Review Feedback at Scale using Large Language Models
Industry Papers
Audrey You University of Auckland, Jingyi (Jenny) Wang University of Auckland, Youxiang Lei Multitudes, Lauren Peate Multitudes, Kelly Blincoe University of Auckland
11:10
20m
Talk
HalluJudge: A Reference-Free Hallucination Detection for Context Misalignment in Code Review Automation
Industry Papers
Kla Tantithamthavorn Monash University, Hong Yi Lin The University of Melbourne, Patanamon Thongtanunam University of Melbourne, Wachiraphan (Ping) Charoenwet University of Melbourne, Minwoo Jeong Atlassian, Ming Wu Atlassian
11:30
20m
Talk
Hydra-Reviewer: A holistic multi-agent system for automatic code review comment generation
Journal-First Paper
Xiaoxue Ren Zhejiang University, Chaoqun Dai Zhejiang Gongshang University, Qiao Huang Zhejiang Gongshang University, Ye Wang Zhejiang Gongshang University, Chao Liu Chongqing University, Bo Jiang Zhejiang Gongshang University
Link to publication
11:50
20m
Talk
AI-Assisted Fixes to Code Review Comments at Scale
Industry Papers
Chandra Sekhar Maddila Meta Platforms, Inc., Negar Ghorbani Meta Platforms Inc., James Saindon Meta, Parth Thakkar Meta Platforms, Inc., Vijayaraghavan Murali Meta Platforms Inc., Rui Abreu Meta, Jingyue Shen Meta Platforms Inc., Brian Zhou Meta Platforms Inc., Nachiappan Nagappan Meta Platforms, Inc., Peter C Rigby Meta / Concordia University
12:10
20m
Talk
Code Reviewer Recommendation for High Risk Diffs at Scale: Workflow, Recommender, and Live Experiments
Industry Papers
Aishwarya Girish Paraspatki Meta Platforms, Inc., Brandon Reznicek Meta Platforms, Inc., Rui Abreu Meta, Ford Garberson Meta Platforms, Inc., Audris Mockus University of Tennessee, Nachiappan Nagappan Meta Platforms, Inc., Peter C Rigby Meta / Concordia University