On the Impact of Code Comments for Automated Bug-Fixing: An Empirical Study
Large Language Models (LLMs) are increasingly relevant in Software Engineering research and practice, with Automated Bug Fixing (ABF) being one of their key applications. ABF involves transforming a buggy method into its fixed equivalent. In this study, we investigate how the presence or absence of comments, both during training and at inference time, impacts the bug-fixing capabilities of LLMs. We conduct an empirical evaluation comparing two model families, each evaluated under all combinations of training and inference conditions (with and without comments), and thereby revisiting the common practice of removing comments during training. To address the limited availability of comments in state-of-the-art datasets, we use an LLM to automatically generate comments for methods lacking them. Our findings show that comments improve ABF accuracy by up to threefold when present in both phases, while training with comments does not degrade performance when instances lack them.
Mon 13 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | Session 5 - Summarization, Documentation, and Code ReviewResearch Track / Vaclav Rajlich Early Career Award / ICPC Program / Journal First at Europa II Chair(s): Masud Rahman Dalhousie University | ||
11:00 10mTalk | Vaclav Rajlich Award Vaclav Rajlich Early Career Award Marvin Wyrich Saarland University | ||
11:10 10mTalk | RepoMind: Enhancing Repository-Level Code Generation via LLM Reasoning over Structured Repository Documentation Research Track Songwen Gong South China University of Technology, Mengzhen Wang South China University of Technology, Jiexin Wang South China University of Technology, Yi Cai School of Software Engineering, South China University of Technology, Guangzhou, China | ||
11:20 10mTalk | SQL-Commenter: Aligning Large Language Models for SQL Comment Generation with Direct Preference Optimization Research Track Lei Yu Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Peng Wang Institute of Information Engineering,Chinese Academy of Sciences, Jingyuan Zhang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, China, Xin Wang Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Jia Xu Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Li Yang Institute of Software, Chinese Academy of Sciences, Changzhi Deng Institute of Software, Chinese Academy of Sciences, Jiajia Ma Institute of Software, Chinese Academy of Sciences, China, Fengjun Zhang Institute of Software, Chinese Academy of Sciences, China Pre-print Media Attached File Attached | ||
11:30 10mTalk | Studying Quality Improvements Recommended via Manual and Automated Code Review Research Track Giuseppe Crupi Università della Svizzera italiana, Rosalia Tufano Università della Svizzera Italiana, Gabriele Bavota Software Institute @ Università della Svizzera Italiana Pre-print | ||
11:40 10mTalk | Towards Universal Segmentation for Log Parsing Research Track Van-Hoang Le University of Luxembourg, Luxembourg, Domenico Bianculli University of Luxembourg, Huy-Trung Nguyen Posts and Telecommunications Institute of Technology Pre-print | ||
11:50 10mTalk | DPS: Design Pattern Summarisation Using Code Features Journal First Najam Nazar Monash University, Sameer Sikka University of Melbourne, Christoph Treude Singapore Management University | ||
12:00 10mTalk | On the Impact of Code Comments for Automated Bug-Fixing: An Empirical Study Research Track Antonio Vitale Politecnico di Torino, University of Molise, Emanuela Guglielmi University of Molise, Simone Scalabrino University of Molise, Rocco Oliveto University of Molise Pre-print | ||
12:10 20mLive Q&A | Joint QA and Discussion ICPC Program | ||