Explainable Fault Localization for Programming Assignments via LLM-Guided Annotation
Providing timely and personalized guidance for students’ programming assignments, particularly by indicating fine-grained error locations with explanations, offers significant practical value for helping students complete assignments and enhance their learning outcomes. In recent years, various automated Fault Localization (FL) techniques, particularly those leveraging Large Language Models (LLMs), have demonstrated promising results in identifying errors in programs. However, existing fault localization techniques face challenges when applied to educational contexts. Most approaches operate at the method-level without explanatory feedback, resulting in granularity too coarse for students who need actionable insights to identify and fix their errors. While some approaches attempt line-level fault localization, they often depend on predicting line numbers directly in numerical form, which is ill-suited to LLMs. To address these challenges, we propose FLAME, a fine-grained, explainable Fault Localization method tailored for programming assignments via LLM-guided Annotation and Model Ensemble. FLAME leverages rich contextual information specific to programming assignments to guide LLMs in identifying faulty code lines. Instead of directly predicting line numbers, we prompt the LLM to annotate faulty code lines with detailed explanations, enhancing both localization accuracy and educational utility. To further improve reliability, we introduce a weighted multi-model voting strategy that aggregates results from multiple LLMs to determine the suspiciousness of each code line. Extensive experimental results demonstrate that FLAME outperforms state-of-the-art fault localization baselines on programming assignments, successfully localizing 207 more faults at top-1 over the best-performing baseline. Beyond educational contexts, FLAME also generalizes effectively to general-purpose software codebases, outperforming all baselines on the Defects4J benchmark.
Tue 18 NovDisplayed time zone: Seoul change
11:00 - 12:30 | Fault LocalizationResearch Papers / Journal-First at Grand Hall 2 Chair(s): Rui Abreu Faculty of Engineering of the University of Porto, Portugal | ||
11:00 10mTalk | FaultSeeker: LLM-Empowered Framework for Blockchain Transaction Fault Localization Research Papers Kairan Sun Nanyang Technological University, Zhengzi Xu Imperial Global Singapore, Kaixuan Li Nanyang Technological University, Lyuye Zhang Nanyang Technological University, Yuqiang Sun Nanyang Technological University, Liwei Tan MetaTrust Labs, Yang Liu Nanyang Technological University | ||
11:10 10mTalk | FlexFL: Flexible and Effective Fault Localization With Open-Source Large Language Models Journal-First Chuyang Xu Zhejiang University, Zhongxin Liu Zhejiang University, Xiaoxue Ren Zhejiang University, Gehao Zhang Ant Group, Ming Liang Ant Group, David Lo Singapore Management University | ||
11:20 10mTalk | LLM-Based Identification of Null Pointer Exception Patches Research Papers Tahir Ullah Beijing Institute of Technology, Waseem Akram Beijing Institute of Technology, Fiza Khaliq Beijing Institute of Technology, Hui Liu Beijing Institute of Technology | ||
11:30 10mTalk | SpectAcle: Fault Localisation of AI-Enabled CPS by Exploiting Sequences of DNN Controller Inferences Journal-First Deyun Lyu National Institute of Informatics, Zhenya Zhang Kyushu University, Paolo Arcaini National Institute of Informatics
, Xiao-Yi Zhang University of Science and Technology Beijing, Fuyuki Ishikawa National Institute of Informatics, Jianjun Zhao Kyushu University Link to publication DOI | ||
11:40 10mTalk | Sifting Truth from Coincidences: A Two-Stage Positive and Unlabeled Learning Model for Coincidental Correctness Detection Research Papers Chunyan Liu Chongqing University, Huan Xie Chongqing University, Yan Lei Chongqing University, Zhenyu Wu School of Big Data & Software Engineering, Chongqing University, Jinping Wang Chonqing University | ||
11:50 10mTalk | Let the Code Speak: Incorporating Program Dynamic State for Better Method-Level Fault Localization Research Papers Yihao Qin , Shangwen Wang National University of Defense Technology, Bo Lin National University of Defense Technology, Xin Peng , Sheng Ouyang National University of Defense Technology, Liqian Chen National University of Defense Technology, Xiaoguang Mao National University of Defense Technology | ||
12:00 10mTalk | Issue Localization via LLM-Driven Iterative Code Graph Searching Research Papers Zhonghao Jiang Zhejiang University, Xiaoxue Ren Zhejiang University, Meng Yan Chongqing University, Wei Jiang Ant Group, Yong Li Ant Group, Zhongxin Liu Zhejiang University | ||
12:10 10mTalk | Hypergraph Neural Network-based Multi-Granular Root Cause Localization for Microservice Systems Research Papers Yaxiao Li Xidian University, Lu Wang Xidian University, Chenxi Zhang Xidian University, Qingshan Li Xidian University, Siming Rong Xidian University, Baiyang Wen Xidian University, Xuyang Li Purdue University, Kun Ma Xidian University, Quanwei Du Xidian University, KeYang Li Xidian University, Lingfeng Pan Xidian University, Xinyue Li Peking University, MingXuan Hui Xidian University | ||
12:20 10mTalk | Explainable Fault Localization for Programming Assignments via LLM-Guided Annotation Research Papers Fang Liu Beihang University, Tianze Wang Beihang University, Li Zhang Beihang University, Zheyu Yang Beihang University, Jing Jiang Beihang University, Zian Sun Beihang University Pre-print | ||