Standing on the Shoulders of Giants: Bug-Aware Automated GUI Testing via Retrieval Augmentation
In software development, similar apps often encounter similar bugs due to shared functionalities and implementation methods. However, current automated GUI testing methods mainly focus on generating test scripts to cover more pages by analyzing the internal structure of the app, without targeted exploration of paths that may trigger bugs, resulting in low efficiency in bug discovery. Considering that a large number of bug reports on open source platforms can provide external knowledge for testing, this paper proposes Bu g Hu n te r, a novel bug-aware automated GUI testing approach that generates exploration paths guided by bug reports from similar apps, utilizing a combination of multimodal large language models (MLLMs) and Retrieval-Augmented Generation (RAG). Instead of focusing solely on coverage, BugHunter dynamically adapts the testing process to target bug paths, thereby increasing bug detection efficiency. BugHunter first builds a high-quality bug knowledge base from historical bug reports. Then it retrieves relevant reports from this large bug knowledge base using a two-stage retrieval process, and generates test paths based on similar apps’ bug reports. BugHunter also introduces a local and global path-planning mechanism to handle differences in functionality and UI design across apps, and the ambiguous behavior or missing steps in the online bug reports. We evaluate BugHunter on 121 bugs across 71 apps and compare its performance against 16 state-of-the-art baselines. BugHunter achieves 60% improvement in bug detection over the best baseline, with comparable or higher coverage against the baselines. Furthermore, bu g successfully detects 49 new crash bugs in real-world apps from Google Play, with 33 bugs fixed, 9 confirmed, and 7 pending feedback.
Tue 24 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30 | UI TestingResearch Papers / Journal First / Industry Papers at Andromeda Chair(s): Chunyang Chen TU Munich | ||
14:00 10mTalk | ProphetAgent: Automatically Synthesizing GUI Tests from Test Cases in Natural Language for Mobile Apps Industry Papers Qichao Kong ByteDance, Zhengwei Lv ByteDance, Yiheng Xiong East China Normal University, Jingling Sun University of Electronic Science and Technology of China, Ting Su East China Normal University, Dingchun Wang ByteDance Ltd, Beijing, China, Letao Li ByteDance Ltd, Beijing, China, Xu Yang ByteDance, Gang Huo ByteDance | ||
14:10 20mTalk | Standing on the Shoulders of Giants: Bug-Aware Automated GUI Testing via Retrieval Augmentation Research Papers Mengzhuo Chen Institute of Software, Chinese Academy of Sciences, Zhe Liu Institute of Software, Chinese Academy of Sciences, Chunyang Chen TU Munich, Junjie Wang Institute of Software at Chinese Academy of Sciences, Boyu Wu University of Chinese Academy of Sciences, Beijing, China, Jun Hu Institute of Software, Chinese Academy of Sciences, Qing Wang Institute of Software at Chinese Academy of Sciences DOI | ||
14:30 20mTalk | A Mixed-Methods Study of Model-Based GUI Testing in Real-World Industrial Settings Research Papers Shaoheng Cao Nanjing University, Renyi Chen Samsung Electronics(China)R&D Centre, Wenhua Yang Nanjing University of Aeronautics and Astronautics, Minxue Pan Nanjing University, Xuandong Li Nanjing University DOI | ||
14:50 20mTalk | Non-Flaky and Nearly-Optimal Time-based Treatment of Asynchronous Wait Web Tests Journal First yu pei , Jeongju Sohn Kyungpook National University, Sarra Habchi Ubisoft Montréal, Mike Papadakis University of Luxembourg | ||
15:10 20mTalk | LLMDroid: Enhancing Automated Mobile App GUI Testing Coverage with Large Language Model Guidance Research Papers Chenxu Wang Huazhong University of Science and Technology, Tianming Liu Monash Univerisity, Yanjie Zhao Huazhong University of Science and Technology, Minghui Yang OPPO, Haoyu Wang Huazhong University of Science and Technology DOI |
Andromeda is located close to the restaurant and the bar, at the end of the corridor on the side of the bar.
From the registration desk, go towards the restaurant, turn left towards the bar, walk until the end of the corridor.