ICSME 2025
Sun 7 - Fri 12 September 2025 Auckland, New Zealand
Wed 10 Sep 2025 13:30 - 13:45 at Case Room 2 260-057 - Session 4 - Testing 1 Chair(s): Sigrid Eldh

Performance testing is crucial to ensuring that web applications meet user expectations under varying workloads. Activities such as stress, load, and smoke testing are designed to simulate different kinds of simultaneous user interactions and assess system behavior. Despite its recognized importance in quality assurance of large-scale web-based systems, witnessed by numerous studies proposing solutions to support these activities, the real-world adoption and evolutionary dynamics of performance tests have received limited attention in the literature. To fill this gap, we analyzed 77 open-source web projects using Apache JMeter and Locust. Our study investigates how performance tasks are performed (adoption time, load design, types of tasks), the characteristics of projects that adopt them, and their long-term maintenance. Our findings reveal that performance tests in open-source projects are simple, with a focus on single-user behaviors and minimal requests, and most tests have low concurrency. Load tests are the most common, followed by smoke and stress tests. Projects with performance tests tend to be larger and more actively maintained. However, tests are mostly long-lived but rarely updated, suggesting potential risks to their relevance and coverage over time. Finally, by creating a taxonomy of performance test changes, we observe recurring patterns of modifications, including workload adjustments, network request changes, and updates to system monitoring.

Wed 10 Sep

Displayed time zone: Auckland, Wellington change

13:30 - 15:00
Session 4 - Testing 1Research Papers Track / Registered Reports / Journal First Track / NIER Track / Industry Track / Tool Demonstration Track at Case Room 2 260-057
Chair(s): Sigrid Eldh Ericsson AB, Mälardalen University, Carleton University
13:30
15m
Performance Testing in Open-Source Web Projects: Adoption, Maintenance, and a Change Taxonomy
Research Papers Track
Sergio Di Meglio Università degli Studi di Napoli Federico II, Luigi Libero Lucio Starace Università degli Studi di Napoli Federico II, Valeria Pontillo Gran Sasso Science Institute, Ruben Opdebeeck Vrije Universiteit Brussel, Coen De Roover Vrije Universiteit Brussel, Sergio Di Martino Università degli Studi di Napoli Federico II
Pre-print
13:45
15m
Harnessing LLMs for Document-Guided Fuzzing of OpenCV Library
Research Papers Track
Bin Duan The University of Queensland, Tarek Mahmud Texas State University, Meiru Che Central Queensland University, Yan Yan University of Illinois Chicago, Naipeng Dong The University of Queensland, Australia, Dan Dongseong Kim The University of Queensland, Guowei Yang University of Queensland
14:00
10m
XTestGen: Natural Language to Maintainable E2E Test Scripts with LLMs
Tool Demonstration Track
Hiroyuki Kirinuki NTT, Masaki Tajima NTT Software Innovation Center, Kei Wakabayashi NTT
File Attached
14:10
10m
Towards Effective Lightweight Test Oracles for Automated Multi-Fault Program RepairOnline
NIER Track
Omar I. Al-Bataineh Gran Sasso Science Institute (GSSI)
14:20
15m
Testing Is Not Boring: Characterizing Challenge in Software Testing Tasks
Industry Track
Davi Gama Hardman CESAR - Recife Center for Advanced Studies and Systems, César França Federal Rural University of Pernambuco (UFRPE), Brody Stuart-Verner University of Calgary, Ronnie de Souza Santos University of Calgary
14:35
15m
Enriching automatic test case generation by extracting relevant test inputs from bug reports
Journal First Track
Wendkuuni Arzouma Marc Christian OUEDRAOGO University of Luxembourg, Laura Plein CISPA Helmholtz Center for Information Security, Abdoul Kader Kaboré University of Luxembourg, Andrew Habib ABB Corporate Research, Germany, Jacques Klein University of Luxembourg, David Lo Singapore Management University, Tegawendé F. Bissyandé University of Luxembourg
14:50
10m
An Empirical Study of Complexity, Heterogeneity, and Compliance of GitHub Actions Workflows
Registered Reports
Edward Abrokwah Department of Computer Science, Trent University, Peterborough, Canada, Taher A. Ghaleb Trent University
Pre-print