Prompting Is All Your Need: Automated Android Bug Replay with Large Language Models
Bug reports are vital for software maintenance that allow users to inform developers of the problems encountered while using the software. As such, researchers have committed considerable resources toward automating bug replay to expedite the process of software maintenance. Nonetheless, the success of current automated approaches is largely dictated by the characteristics and quality of bug reports, as they are constrained by the limitations of manually-crafted patterns and pre-defined vocabulary lists. Inspired by the success of Large Language Models (LLMs) in natural language understanding, we propose AdbGPT, a new lightweight approach to automatically reproduce the bugs from bug reports through prompt engineering, without any training and hard-coding effort. AdbGPT leverages few-shot learning and chain-of-thought reasoning to elicit human knowledge and logical reasoning from LLMs to accomplish the bug replay in a manner similar to a developer. Our evaluations demonstrate the effectiveness and efficiency of our AdbGPT to reproduce 81.3% of bug reports in 253.6 seconds, significantly outperforming the state-of-the-art baselines and ablation studies. We also conduct a small-scale user study to confirm the usefulness of AdbGPT in enhancing developers’ bug replay capabilities.
Fri 19 AprDisplayed time zone: Lisbon change
11:00 - 12:30 | Testing with and for AI 1Research Track / Journal-first Papers / Demonstrations at Sophia de Mello Breyner Andresen Chair(s): Peter Rigby Concordia University; Meta | ||
11:00 15mTalk | Prompting Is All Your Need: Automated Android Bug Replay with Large Language Models Research Track | ||
11:15 15mTalk | Towards Reliable AI: Adequacy Metrics for Ensuring the Quality of System-level Testing of Autonomous Vehicles Research Track | ||
11:30 15mTalk | Learning-based Widget Matching for Migrating GUI Test Cases Research Track Yakun Zhang Peking University, Wenjie Zhang Peking University, Dezhi Ran Peking University, Qihao Zhu Peking University, Chengfeng Dou Peking University, Dan Hao Peking University, Tao Xie Peking University, Lu Zhang Peking University | ||
11:45 7mTalk | A Search-Based Testing Approach for Deep Reinforcement Learning Agents Journal-first Papers Amirhossein Zolfagharian University of Ottawa - School of Electrical Engineering & Computer Science (EECS), Manel Abdellatif Software and Information Technology Engineering Department, École de Technologie Supérieure, Mojtaba Bagherzadeh Cisco, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland, Ramesh S | ||
11:52 7mTalk | StubCoder: Automated Generation and Repair of Stub Code for Mock Objects Journal-first Papers Hengcheng Zhu The Hong Kong University of Science and Technology, Lili Wei McGill University, Valerio Terragni University of Auckland, Yepang Liu Southern University of Science and Technology, Shing-Chi Cheung Hong Kong University of Science and Technology, Jiarong Wu , Qin Sheng WeBank Co Ltd, Bing Zhang WeBank Co. Ltd., Lihong Song WeBank Co. Ltd. Link to publication DOI Authorizer link Pre-print | ||
11:59 7mTalk | Testing of Deep Reinforcement Learning Agents with Surrogate Models Journal-first Papers | ||
12:06 7mTalk | Model vs System Level Testing of Autonomous Driving Systems: A Replication and Extension Study Journal-first Papers Andrea Stocco Technical University of Munich, fortiss, Brian Pulfer University of Geneva, Paolo Tonella USI Lugano | ||
12:13 7mTalk | SAFE: Safety Analysis and Retraining of DNNs Demonstrations Mohammed Attaoui University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland Pre-print | ||
12:20 7mTalk | MutaBot: A Mutation Testing Approach for Chatbots Demonstrations Michael Ferdinando Urrico University of Milano - Bicocca, Diego Clerissi University of Milano-Bicocca, Leonardo Mariani University of Milano-Bicocca DOI Pre-print Media Attached | ||