ProphetAgent: Automatically Synthesizing GUI Tests from Test Cases in Natural Language for Mobile Apps
GUI tests is crucial for ensuring software quality and user satisfaction of mobile apps. In practice, companies often maintain extensive test cases written in natural language. Testers need to convert these test cases into executable scripts for regression and compatibility testing. Requirement changes or version updates often necessitate the addition and modification to these test cases. Thus, when faced with large volumes of test cases and regular updates, this process becomes costly, which is a common challenge across the industry. To address this issue, this paper proposes ProphetAgent that can automatically synthesize executable GUI tests from the test cases written in natural language. % , and then execute the tests on the corresponding apps. ProphetAgent first constructs a Clustered UI Transition Graph (CUTG) enriched with semantic information, then leverages large language models to generate the executable test case based on CUTG and test cases written in natural language. Experiment results show that ProphetAgent achieved a 78.1% success rate across 120 test cases in Douyin, Doubao, and six open-source apps, surpassing existing automated approaches (21.4% for AppAgent and 32.5% for AutoDroid). Additionally, statistical data from ByteDance’s testing platform reveal that before implementing ProphetAgent, testers managed an average of 25 test cases per day, but this number has increased to 90 with ProphetAgent.
Tue 24 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30 | UI TestingResearch Papers / Journal First / Industry Papers at Andromeda Chair(s): Chunyang Chen TU Munich | ||
14:00 10mTalk | ProphetAgent: Automatically Synthesizing GUI Tests from Test Cases in Natural Language for Mobile Apps Industry Papers Qichao Kong ByteDance, Zhengwei Lv ByteDance, Yiheng Xiong East China Normal University, Jingling Sun University of Electronic Science and Technology of China, Ting Su East China Normal University, Dingchun Wang ByteDance Ltd, Beijing, China, Letao Li ByteDance Ltd, Beijing, China, Xu Yang ByteDance, Gang Huo ByteDance | ||
14:10 20mTalk | Standing on the Shoulders of Giants: Bug-Aware Automated GUI Testing via Retrieval Augmentation Research Papers Mengzhuo Chen Institute of Software, Chinese Academy of Sciences, Zhe Liu Institute of Software, Chinese Academy of Sciences, Chunyang Chen TU Munich, Junjie Wang Institute of Software at Chinese Academy of Sciences, Boyu Wu University of Chinese Academy of Sciences, Beijing, China, Jun Hu Institute of Software, Chinese Academy of Sciences, Qing Wang Institute of Software at Chinese Academy of Sciences DOI | ||
14:30 20mTalk | A Mixed-Methods Study of Model-Based GUI Testing in Real-World Industrial Settings Research Papers Shaoheng Cao Nanjing University, Renyi Chen Samsung Electronics(China)R&D Centre, Wenhua Yang Nanjing University of Aeronautics and Astronautics, Minxue Pan Nanjing University, Xuandong Li Nanjing University DOI | ||
14:50 20mTalk | Non-Flaky and Nearly-Optimal Time-based Treatment of Asynchronous Wait Web Tests Journal First yu pei , Jeongju Sohn Kyungpook National University, Sarra Habchi Ubisoft Montréal, Mike Papadakis University of Luxembourg | ||
15:10 20mTalk | LLMDroid: Enhancing Automated Mobile App GUI Testing Coverage with Large Language Model Guidance Research Papers Chenxu Wang Huazhong University of Science and Technology, Tianming Liu Monash Univerisity, Yanjie Zhao Huazhong University of Science and Technology, Minghui Yang OPPO, Haoyu Wang Huazhong University of Science and Technology DOI |
Andromeda is located close to the restaurant and the bar, at the end of the corridor on the side of the bar.
From the registration desk, go towards the restaurant, turn left towards the bar, walk until the end of the corridor.