FlexFL: Flexible and Effective Fault Localization With Open-Source Large Language Models
This program is tentative and subject to change.
Fault localization (FL) targets identifying bug locations within a software system, which can enhance debugging efficiency and improve software quality. Due to the impressive code comprehension ability of Large Language Models (LLMs), a few studies have proposed to leverage LLMs to locate bugs, i.e., LLM-based FL, and demonstrated promising performance. However, first, these methods are limited in flexibility. They rely on bug-triggering test cases to perform FL and cannot make use of other available bug-related information, e.g., bug reports. Second, they are built upon proprietary LLMs, which are, although powerful, confronted with risks in data privacy. To address these limitations, we propose a novel LLM-based FL framework named FlexFL, which can flexibly leverage different types of bug-related information and effectively work with open-source LLMs. FlexFL is composed of two stages. In the first stage, FlexFL reduces the search space of buggy code using state-of-the-art FL techniques of different families and provides a candidate list of bug-related methods. In the second stage, FlexFL leverages LLMs to delve deeper to double-check the code snippets of methods suggested by the first stage and refine fault localization results. In each stage, FlexFL constructs agents based on open-source LLMs, which share the same pipeline that does not postulate any type of bug-related information and can interact with function calls without the out-of-the-box capability. Extensive experimental results on Defects4J demonstrate that FlexFL outperforms the baselines and can work with different open-source LLMs. Specifically, FlexFL with a lightweight open-source LLM Llama3-8B can locate 42 and 63 more bugs than two state-of-the-art LLM-based FL approaches AutoFL and AgentFL that both use GPT-3.5. In addition, FlexFL can localize 93 bugs that cannot be localized by non-LLM-based FL techniques at the top 1. Furthermore, to mitigate potential data contamination, we conduct experiments on a dataset which Llama3-8B has not seen before, and the evaluation results show that FlexFL can also achieve good performance.
This program is tentative and subject to change.
Tue 18 NovDisplayed time zone: Seoul change
11:00 - 12:30 | |||
11:00 10mTalk | FaultSeeker: LLM-Empowered Framework for Blockchain Transaction Fault Localization Research Papers Kairan Sun Nanyang Technological University, Zhengzi Xu Imperial Global Singapore, Kaixuan Li Nanyang Technological University, Lyuye Zhang Nanyang Technological University, Yuqiang Sun Nanyang Technological University, Liwei Tan MetaTrust Labs, Yang Liu Nanyang Technological University | ||
11:10 10mTalk | FlexFL: Flexible and Effective Fault Localization With Open-Source Large Language Models Journal-First Track Chuyang Xu Zhejiang University, Zhongxin Liu Zhejiang University, Xiaoxue Ren Zhejiang University, Gehao Zhang Ant Group, Ming Liang Ant Group, David Lo Singapore Management University | ||
11:20 10mTalk | LLM-Based Identification of Null Pointer Exception Patches Research Papers Tahir Ullah Beijing Institute of Technology, Waseem Akram Beijing Institute of Technology, Fiza Khaliq Beijing Institute of Technology, Hui Liu Beijing Institute of Technology | ||
11:30 10mTalk | SpectAcle: Fault Localisation of AI-Enabled CPS by Exploiting Sequences of DNN Controller Inferences Journal-First Track Deyun Lyu National Institute of Informatics, Zhenya Zhang Kyushu University, Japan, Paolo Arcaini National Institute of Informatics
, Xiao-Yi Zhang University of Science and Technology Beijing, Fuyuki Ishikawa National Institute of Informatics, Jianjun Zhao Kyushu University | ||
11:40 10mTalk | Sifting Truth from Coincidences: A Two-Stage Positive and Unlabeled Learning Model for Coincidental Correctness Detection Research Papers Chunyan Liu Chongqing University, Huan Xie Chongqing University, Yan Lei Chongqing University, Zhenyu Wu School of Big Data & Software Engineering, Chongqing University, Jinping Wang Chonqing University | ||
11:50 10mTalk | Let the Code Speak: Incorporating Program Dynamic State for Better Method-Level Fault Localization Research Papers Yihao Qin , Shangwen Wang National University of Defense Technology, Bo Lin National University of Defense Technology, Xin Peng , Sheng Ouyang National University of Defense Technology, Liqian Chen National University of Defense Technology, Xiaoguang Mao National University of Defense Technology | ||
12:00 10mTalk | Issue Localization via LLM-Driven Iterative Code Graph Searching Research Papers Zhonghao Jiang Zhejiang University, Xiaoxue Ren Zhejiang University, Meng Yan Chongqing University, Wei Jiang Ant Group, Yong Li Ant Group, Zhongxin Liu Zhejiang University | ||
12:10 10mTalk | Hypergraph Neural Network-based Multi-Granular Root Cause Localization for Microservice Systems Research Papers Yaxiao Li Xidian University, Lu Wang Xidian University, Chenxi Zhang Xidian University, Qingshan Li Xidian University, Siming Rong Xidian University, Baiyang Wen Xidian University, Xuyang Li Purdue University, Kun Ma Xidian University, Quanwei Du Xidian University, KeYang Li Xidian University, Lingfeng Pan Xidian University, Xinyue Li Peking University, MingXuan Hui Xidian University | ||
12:20 10mTalk | Explainable Fault Localization for Programming Assignments via LLM-Guided Annotation Research Papers Fang Liu Beihang University, Tianze Wang Beihang University, Li Zhang Beihang University, Zheyu Yang Beihang University, Jing Jiang Beihang University, Zian Sun Beihang University Pre-print | ||