A Mixed-Methods Study of Model-Based GUI Testing in Real-World Industrial Settings
Model-based testing (MBT) has been an important methodology in software engineering, attracting extensive research attention for over four decades. However, despite its academic acclaim, studies examining the impact of MBT in industrial environments—particularly regarding its long-term effects—remain limited and yield unclear results. This gap may contribute to the challenges in establishing a study environment for implementing and applying MBT in production settings to evaluate its impact over time. To bridge this gap, we collaborated with an industrial partner to undertake a comprehensive, longitudinal empirical study employing mixed methods. Over two months, we implemented our MBT tool within the corporation, assessing the short- and long-term effectiveness and efficiency of MBT compared to script-writing-based testing. Through a mix of quantitative and qualitative methods—spanning experimental data, questionnaire surveys, and interviews—our study uncovers several insightful findings. These include differences in effectiveness and maintainability between short- and long-term MBT application, the evolving perceptions and expectations of engineers regarding MBT, and more. Leveraging these insights, we propose actionable implications for both the academic and industrial communities, aimed at bolstering confidence in MBT adoption and investment for software testing purposes.
Tue 24 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30 | UI TestingResearch Papers / Journal First / Industry Papers at Andromeda Chair(s): Chunyang Chen TU Munich | ||
14:00 10mTalk | ProphetAgent: Automatically Synthesizing GUI Tests from Test Cases in Natural Language for Mobile Apps Industry Papers Qichao Kong ByteDance, Zhengwei Lv ByteDance, Yiheng Xiong East China Normal University, Jingling Sun University of Electronic Science and Technology of China, Ting Su East China Normal University, Dingchun Wang ByteDance Ltd, Beijing, China, Letao Li ByteDance Ltd, Beijing, China, Xu Yang ByteDance, Gang Huo ByteDance | ||
14:10 20mTalk | Standing on the Shoulders of Giants: Bug-Aware Automated GUI Testing via Retrieval Augmentation Research Papers Mengzhuo Chen Institute of Software, Chinese Academy of Sciences, Zhe Liu Institute of Software, Chinese Academy of Sciences, Chunyang Chen TU Munich, Junjie Wang Institute of Software at Chinese Academy of Sciences, Boyu Wu University of Chinese Academy of Sciences, Beijing, China, Jun Hu Institute of Software, Chinese Academy of Sciences, Qing Wang Institute of Software at Chinese Academy of Sciences DOI | ||
14:30 20mTalk | A Mixed-Methods Study of Model-Based GUI Testing in Real-World Industrial Settings Research Papers Shaoheng Cao Nanjing University, Renyi Chen Samsung Electronics(China)R&D Centre, Wenhua Yang Nanjing University of Aeronautics and Astronautics, Minxue Pan Nanjing University, Xuandong Li Nanjing University DOI | ||
14:50 20mTalk | Non-Flaky and Nearly-Optimal Time-based Treatment of Asynchronous Wait Web Tests Journal First yu pei , Jeongju Sohn Kyungpook National University, Sarra Habchi Ubisoft Montréal, Mike Papadakis University of Luxembourg | ||
15:10 20mTalk | LLMDroid: Enhancing Automated Mobile App GUI Testing Coverage with Large Language Model Guidance Research Papers Chenxu Wang Huazhong University of Science and Technology, Tianming Liu Monash Univerisity, Yanjie Zhao Huazhong University of Science and Technology, Minghui Yang OPPO, Haoyu Wang Huazhong University of Science and Technology DOI |
Andromeda is located close to the restaurant and the bar, at the end of the corridor on the side of the bar.
From the registration desk, go towards the restaurant, turn left towards the bar, walk until the end of the corridor.