ICST 2025
Mon 31 March - Fri 4 April 2025 Naples, Italy

This program is tentative and subject to change.

Wed 2 Apr 2025 12:15 - 12:30 at Room A - Testing, Localisation and Repair Chair(s): Shin Yoo

Abstract—Continuous Integration and Continuous Delivery (CI/CD) processes are vital to meet the growing demands of open source software (OSS), providing a pipeline to enhance project quality and productivity. To ensure the user interfaces (UIs) of these systems work as intended, UI testing is crucial for verifying visual elements of software. Integrating UI tests in CI/CD pipelines should provide fast delivery and comprehensive test coverage. However, there is a gap in understanding how popular UI testing frameworks are adopted within CI/CD workflows— and the effects of this integration on OSS development. Aims: This study aims to explore developers’ perceptions of the challenges, strategies, and impacts of incorporating UI testing into CI/CD environments. In particular, we focus on OSS developers utilizing popular web-based UI testing frameworks— such as Selenium, Cypress, and Playwright—and popular CI/CD platforms—including GitHub Actions, Travis CI, CircleCI, and Jenkins—on public GitHub repositories. Method: We conducted an online survey targeting OSS de- velopers (n = 94) from GitHub with experience integrating UI testing frameworks into configuration files for CI/CD platforms. To augment our results, we conducted follow-up interviews (n = 18) to gain insights on the challenges, opportunities, and impacts of integrating UI testing into CI/CD pipelines. Results: Our results indicate adapting testing strategy, flaki- ness and longer executions are major challenges in integrating UI testing into CI pipelines—negatively impacting development practices. Alternatively, the benefits include support for realistic test cases and increased detection of issues. However, developers lack effective strategies to mitigate the challenges, relying on ad hoc trial-and-error based approaches, such as temporarily removing flaky tests from CI workflows until they are resolved. Conclusion: Our findings provide implications for OSS devel- opers working on or considering including UI tests in CI/CD pipelines. We also motivate future directions for research and tooling to improve UI testing integration in CI/CD workflows.

This program is tentative and subject to change.

Wed 2 Apr

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:00 - 12:30
Testing, Localisation and RepairJournal-First Papers / Research Papers / Industry at Room A
Chair(s): Shin Yoo Korea Advanced Institute of Science and Technology
11:00
15m
Talk
A Taxonomy of Integration-relevant Faults for Microservice Testing
Research Papers
Lena Gregor Technical University of Munich, Anja Hentschel Siemens AG, Leon Kastner Technical University of Munich, Alexander Pretschner TU Munich
11:15
15m
Talk
Suspicious Types and Bad Neighborhoods- Filtering Spectra with Compiler Information
Research Papers
Leonhard Applis Delft University of Technology, Matthías Páll Gissurarson Chalmers University of Technology, Sweden, Annibale Panichella Delft University of Technology
11:30
15m
Talk
Summary of Automated Program Repair for Variability Bugs in Software Product Line Systems
Journal-First Papers
Thu-Trang Nguyen , Xiao-Yi Zhang University of Science and Technology Beijing, Paolo Arcaini National Institute of Informatics , Fuyuki Ishikawa National Institute of Informatics, Hieu Vo VNU University of Engineering and Technology
Link to publication DOI
11:45
15m
Talk
Automated Engineering of Domain-Specific Metamorphic Testing Environments
Journal-First Papers
Pablo Gómez-Abajo Universidad Autónoma de Madrid, Pablo C Canizares Autonomous University of Madrid, Spain, Alberto Núňez University Complutense of Madrid, Spain., Esther Guerra Universidad Autónoma de Madrid, Juan de Lara Autonomous University of Madrid
Link to publication DOI
12:00
15m
Talk
Speculative Testing at Google with Transition Prediction
Industry
12:15
15m
Talk
Challenges, Strategies, and Impacts: A Qualitative Study on UI Testing in CI/CD Processes from GitHub Developers’ Perspectives
Research Papers
XIAOXIAO GAN Virginia Tech, Huayu Liang Virginia Tech, Chris Brown Virginia Tech