LLMDroid: Enhancing Automated Mobile App GUI Testing Coverage with Large Language Model Guidance
With the rapid development of Large Language Models (LLMs), their integration into automated mobile GUI testing has emerged as a promising research direction. However, existing LLM-based testing approaches face significant challenges, including time inefficiency and high costs due to constant LLM querying. To address these issues, this paper introduces LLMDroid, a novel testing framework designed to enhance existing automated mobile GUI testing tools by leveraging LLMs more efficiently. The workflow of LLMDroid comprises two main stages: Autonomous Exploration and LLM Guidance. During Autonomous Exploration, LLMDroid utilizes existing testing tools while leveraging LLMs to summarize explored pages. When code coverage growth slows, it transitions to LLM Guidance to strategically direct testing towards unexplored functionalities. This approach minimizes LLM interactions while maximizing their impact on test coverage. We applied LLMDroid to three popular open-source Android testing tools and evaluated it on 14 top-listed apps from Google Play. Results demonstrate an average increase of 26.16% in code coverage and 29.31% in activity coverage. Furthermore, our cost analysis reveals that LLMDroid achieves optimal performance at $4.77 per hour using GPT-4o, with a cost-effective alternative achieving 78% of optimal performance at just $0.18 per hour. These findings highlight LLMDroid’s effectiveness in enhancing automated mobile app testing and its potential for widespread adoption.
Tue 24 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30 | UI TestingResearch Papers / Journal First / Industry Papers at Andromeda Chair(s): Chunyang Chen TU Munich | ||
14:00 10mTalk | ProphetAgent: Automatically Synthesizing GUI Tests from Test Cases in Natural Language for Mobile Apps Industry Papers Qichao Kong ByteDance, Zhengwei Lv ByteDance, Yiheng Xiong East China Normal University, Jingling Sun University of Electronic Science and Technology of China, Ting Su East China Normal University, Dingchun Wang ByteDance Ltd, Beijing, China, Letao Li ByteDance Ltd, Beijing, China, Xu Yang ByteDance, Gang Huo ByteDance | ||
14:10 20mTalk | Standing on the Shoulders of Giants: Bug-Aware Automated GUI Testing via Retrieval Augmentation Research Papers Mengzhuo Chen Institute of Software, Chinese Academy of Sciences, Zhe Liu Institute of Software, Chinese Academy of Sciences, Chunyang Chen TU Munich, Junjie Wang Institute of Software at Chinese Academy of Sciences, Boyu Wu University of Chinese Academy of Sciences, Beijing, China, Jun Hu Institute of Software, Chinese Academy of Sciences, Qing Wang Institute of Software at Chinese Academy of Sciences DOI | ||
14:30 20mTalk | A Mixed-Methods Study of Model-Based GUI Testing in Real-World Industrial Settings Research Papers Shaoheng Cao Nanjing University, Renyi Chen Samsung Electronics(China)R&D Centre, Wenhua Yang Nanjing University of Aeronautics and Astronautics, Minxue Pan Nanjing University, Xuandong Li Nanjing University DOI | ||
14:50 20mTalk | Non-Flaky and Nearly-Optimal Time-based Treatment of Asynchronous Wait Web Tests Journal First yu pei , Jeongju Sohn Kyungpook National University, Sarra Habchi Ubisoft Montréal, Mike Papadakis University of Luxembourg | ||
15:10 20mTalk | LLMDroid: Enhancing Automated Mobile App GUI Testing Coverage with Large Language Model Guidance Research Papers Chenxu Wang Huazhong University of Science and Technology, Tianming Liu Monash Univerisity, Yanjie Zhao Huazhong University of Science and Technology, Minghui Yang OPPO, Haoyu Wang Huazhong University of Science and Technology DOI |
Andromeda is located close to the restaurant and the bar, at the end of the corridor on the side of the bar.
From the registration desk, go towards the restaurant, turn left towards the bar, walk until the end of the corridor.