LineBreaker: Finding Token-Inconsistency Bugs using Large Language Models
This program is tentative and subject to change.
Token-inconsistency bugs (TIBs) involve the misuse of syntactically valid yet incorrect code tokens, such as misused variables and erroneous function invocations, which can often lead to software bugs. Unlike simple syntactic bugs, TIBs occur at the semantic level and are subtle - sometimes remain undetected for years. Traditional detection methods, such as static analysis and dynamic testing, often struggle with TIBs due to their versatile and context-dependent nature. However, advancements in large language models (LLMs) like GPT-4 present new opportunities for automating TIB detection by leveraging these models’ semantic understanding capabilities.
This paper reports the first systematic measurement of LLMs’ capabilities in detecting TIBs, revealing that while GPT-4 shows promise, it exhibits limitations in precision and scalability. Specifically, its detection capability is undermined by the model’s tendency to focus on the code snippets that do not contain TIBs; its scalability concern arises from GPT-4’s high cost and the massive amount of code requiring inspection. To address these challenges, we introduce LineBreaker, a novel and cascaded TIB detection system. LineBreaker leverages smaller, code-specific, and highly efficient language models to filter out large numbers of code snippets unlikely to contain TIBs, thereby significantly enhancing the system’s performance in terms of precision, recall, and scalability. We evaluated LineBreaker on 154 Python and C GitHub repositories, each with over 1,000 stars, uncovering 123 new flaws, 45% of which could be exploited to disrupt program functionalities. Out of our 69 submitted fixes, 41 have already been confirmed or merged
This program is tentative and subject to change.
Mon 17 NovDisplayed time zone: Seoul change
11:00 - 12:30 | |||
11:00 10mTalk | The Fault in our Stats Research Papers | ||
11:10 10mTalk | Agents in the Sandbox: End-to-End Crash Bug Reproduction for Minecraft Research Papers Eray Yapağcı Bilkent University, Yavuz Alp Sencer Öztürk Bilkent University, Eray Tüzün Bilkent University | ||
11:20 10mTalk | Finding Bugs in MLIR Compiler Infrastructure via Lowering Space Exploration Research Papers Jingjing Liang East China Normal University, Shan Huang East China Normal University, Ting Su East China Normal University | ||
11:30 10mTalk | Why Do Machine Learning Notebooks Crash? An Empirical Study on Public Python Jupyter Notebooks Journal-First Track Yiran Wang Linköping University, Willem Meijer Linköping University, José Antonio Hernández López Universidad de Murcia, Ulf Nilsson Linköping University, Daniel Varro Linköping University / McGill University | ||
11:40 10mTalk | When AllClose Fails: Round-Off Error Estimation for Deep Learning Programs Research Papers Qi Zhan Zhejiang University, Xing Hu Zhejiang University, Yuanyi Lin Huawei Technologies, Tongtong Xu Huawei, Xin Xia Zhejiang University, Shanping Li Zhejiang University | ||
11:50 10mTalk | LLM-Powered Multi-Agent Collaboration for Intelligent Industrial On-Call Automation Research Papers Ruowei Fu Nankai University, Yang Zhang ByteDance Inc., Zeyu Che Nankai University, Xin Wu ByteDance Inc., Zhenyu Zhong Nankai University, Zhiqiang Ren ByteDance Inc., Shenglin Zhang Nankai University, Feng Wang ByteDance Inc., Yongqian Sun Nankai University, Xiaozhou Liu ByteDance Inc., Kexin Liu Nankai University, Yu Zhang ByteDance Inc. | ||
12:00 10mTalk | SSR: Safeguarding Staking Rewards by Defining and Detecting Logical Defects in DeFi Staking Research Papers Zewei Lin Sun Yat-sen University, Jiachi Chen Sun Yat-sen University, Jingwen Zhang School of Software Engineering, Sun Yat sen University, Zexu Wang Sun Yat-sen University, Yuming Feng Peng Cheng Laboratory, Weizhe Zhang Harbin Institute of Technology, Zibin Zheng Sun Yat-sen University | ||
12:10 10mTalk | Finding Bugs in WebAssembly Interface Type Binding Generators Research Papers | ||
12:20 10mTalk | LineBreaker: Finding Token-Inconsistency Bugs using Large Language Models Research Papers Hongbo Chen Indiana University Bloomington, Yifan Zhang San Diego State University, Xing Han The Hong Kong University of Science and Technology, Tianhao Mao Indiana University, Huanyao Rong Indiana University Bloomington, Yuheng Zhang Tsinghua University, Hang Zhang Indiana University, XiaoFeng Wang ACM member, Luyi Xing Indiana University Bloomington/University of Illinois Urbana-Champaign, Xun Chen Samsung Research America | ||