Defects4C: Benchmarking Large Language Model Repair Capability with C/C++ Bugs
This program is tentative and subject to change.
Automated Program Repair (APR) plays a critical role in enhancing the quality and reliability of software systems. While substantial progress has been made in Java-based APR, largely facilitated by benchmarks like Defects4J, there remains a significant gap in research on C/C++ program repair, despite the widespread use of C/C++ and the prevalence of associated vulnerabilities. This gap is primarily due to the lack of high-quality, open-source benchmarks tailored for C/C++.
To address this issue, we introduce \textbf{\textit{Defects4C}}, a comprehensive and executable benchmark specifically designed for C/C++ program repair. Our dataset is constructed from real-world C/C++ repositories and includes a large collection of bug-relevant commits (\textbf{9M} in total), \textbf{248} high-quality buggy functions, and \textbf{102} vulnerable functions, all paired with test cases for reproduction. These resources enable rigorous evaluation of repair techniques and support the retraining of learning-based approaches for enhanced performance.
Using \textbf{\textit{Defects4C}}, we conduct a comprehensive empirical study evaluating the effectiveness of \textbf{24} state-of-the-art large language models (LLMs) in repairing C/C++ faults. Our findings offer valuable insights into the strengths and limitations of current LLM-based APR techniques in this domain, highlighting both the need for more robust methods and the critical role of Defects4C in advancing future research.
This program is tentative and subject to change.
Mon 17 NovDisplayed time zone: Seoul change
11:00 - 12:30 | |||
11:00 10mTalk | Defects4C: Benchmarking Large Language Model Repair Capability with C/C++ Bugs Research Papers Jian Wang Nanyang Technological University, Xiaofei Xie Singapore Management University, Qiang Hu Tianjin University, Shangqing Liu Nanjing University, Jiongchi Yu Singapore Management University, Jiaolong Kong Singapore Management University, Yi Li Nanyang Technological University | ||
11:10 10mTalk | MORepair: Teaching LLMs to Repair Code via Multi-Objective Fine-Tuning Journal-First Track Boyang Yang Yanshan University; Beijing JudaoYouda Network Technology, Haoye Tian Aalto University, Jiadong Ren Yanshan University, Hongyu Zhang Chongqing University, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg, Claire Le Goues Carnegie Mellon University, Shunfu Jin Yanshan University Link to publication DOI Pre-print | ||
11:20 10mTalk | When Fine-Tuning LLMs Meets Data Privacy: An Empirical Study of Federated Learning in LLM-Based Program Repair Journal-First Track Wenqiang LUO City University of Hong Kong, Jacky Keung City University of Hong Kong, Boyang Yang Yanshan University; Beijing JudaoYouda Network Technology, He Ye University College London (UCL), Claire Le Goues Carnegie Mellon University, Tegawendé F. Bissyandé University of Luxembourg, Haoye Tian Aalto University, Xuan-Bach D. Le University of Melbourne | ||
11:30 10mTalk | Test-based Patch Clustering for Automatically-Generated Patches Assessment Journal-First Track Matias Martinez Universitat Politècnica de Catalunya (UPC), Maria Kechagia National and Kapodistrian University of Athens, Anjana Perera Oracle Labs, Australia, Justyna Petke University College London, Federica Sarro University College London, Aldeida Aleti Monash University | ||
11:40 10mTalk | Hierarchical Knowledge Injection for Improving LLM-based Program Repair Research Papers Ramtin Ehsani Drexel University, Esteban Parra Rodriguez Belmont University, Sonia Haiduc Florida State University, Preetha Chatterjee Drexel University, USA | ||
11:50 10mTalk | Characterizing Multi-Hunk Patches: Divergence, Proximity, and LLM Repair Challenges Research Papers Noor Nashid University of British Columbia, Daniel Ding University of British Columbia, Keheliya Gallaba Centre for Software Excellence, Ahmed E. Hassan Queen’s University, Ali Mesbah University of British Columbia | ||
12:00 10mTalk | Reinforcement Learning for Mutation Operator Selection in Automated Program Repair Journal-First Track Carol Hanna University College London, Aymeric Blot University of Rennes, IRISA / INRIA, Justyna Petke University College London | ||
12:10 10mTalk | APRMCTS: Improving LLM-based Automated Program Repair with Iterative Tree Search Research Papers Haichuan Hu Nanjing University of Science and Technology, Congqing He School of Computer Sciences, Universiti Sains Malaysia, Xiaochen Xie Department of Management, Zhejiang University, China, Hao Zhang School of Computer Sciences, Universiti Sains Malaysia, Quanjun Zhang School of Computer Science and Engineering, Nanjing University of Science and Technology | ||
12:20 10mTalk | Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Repair Research Papers Kai Huang Technical University of Munich, Jian Zhang Nanyang Technological University, Xiaofei Xie Singapore Management University, Chunyang Chen TU Munich | ||