Not All Input Helps: What Information Should We Feed to LLMs for Vulnerability Repair?
Software vulnerabilities pose significant security risks, and timely repair is crucial to mitigate potential exploits. Large Language Models (LLMs) have shown promise in automating vulnerability repair, but their effectiveness heavily depends on the input information provided. This paper systematically investigates the impact of different input types on LLM-based vulnerability repair performance. We survey 26 recent studies to categorize the input information used, including vulnerable code snippets, vulnerable line markers, CVE/CWE identifiers and descriptions, and additional metadata such as patch context. Through extensive experiments, we analyze how these inputs, individually and in combination, influence the accuracy of vulnerability repair across various LLM architectures and datasets. Our findings reveal that not all input types contribute positively to repair performance; some may introduce noise or redundancy that hinders the model’s ability to generate effective patches. Based on our analysis, we provide practical guidelines for selecting and structuring input information to optimize LLM-based vulnerability repair.
Fri 17 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | AI for Software Engineering 21Research Track / New Ideas and Emerging Results (NIER) / Journal-first Papers at Asia IV Chair(s): Rui Abreu Faculty of Engineering of the University of Porto, Portugal | ||
11:00 15mTalk | On the Evaluation of Large Language Models in Multilingual Vulnerability Repair Journal-first Papers Dong Wang Tianjin University, Junji Yu Tianjin University, Honglin Shu Kyushu University, Michael Fu The University of Melbourne, Kla Tantithamthavorn Monash University, Yasutaka Kamei Kyushu University, Junjie Chen Tianjin University | ||
11:15 15mTalk | Not All Input Helps: What Information Should We Feed to LLMs for Vulnerability Repair? New Ideas and Emerging Results (NIER) DOI | ||
11:30 15mTalk | EMC: A Semantic-Enhanced Malware Classification Method with Robustness and Scalability Research Track Haojun Zhao Huazhong University of Science and Technology, Yueming Wu Huazhong University of Science and Technology, Zhen Li Huazhong University of Science and Technology, Deqing Zou Huazhong University of Science and Technology Media Attached | ||
11:45 15mTalk | When AI Takes the Wheel: Security Analysis of Framework-Constrained Program Generation Research Track Yue Liu Monash University, Zhenchang Xing CSIRO's Data61, Shidong Pan Columbia University & New York University, Kla Tantithamthavorn Monash University Pre-print | ||
12:00 15mTalk | Software Vulnerability Management in the Era of Artificial Intelligence: An Industry Perspective Research Track M. Mehdi Kholoosi Adelaide University, Triet Le Adelaide University, Muhammad Ali Babar School of Computer Science, The University of Adelaide Pre-print | ||
12:15 15mTalk | Towards Scalable and Interpretable Mobile App Risk Analysis via Large Language Models Research Track Yu Yang Zhejiang University, Zhenyuan Li Zhejiang University, Xiandong Ran Huawei Technologies Co., Ltd., Jiahao Liu National University of Singapore, Jiahui Wang Zhejiang University, Bo Yu National University of Defense Technology, Shouling Ji Zhejiang University Media Attached File Attached | ||