XP 2025
Mon 2 - Thu 5 June 2025 Brugg - Windisch, Switzerland
Wed 4 Jun 2025 15:10 - 15:30 at 5.1A17 (Session) - Testing Chair(s): Per Beining

Previous research has identified mutation testing as a promising technique for detecting unintended changes in test behavior during test code refactoring. Despite its theoretical support, the practical adoption of this approach has been hindered by a lack of corresponding tools. Consequently, these studies have been unable to fully validate the effectiveness of mutation testing across various test refactoring scenarios, leaving an in-depth empirical validation open for future research. To address this gap, this study examines MeteoR, a tool developed as a reference implementation to support test refactoring by using mutation testing as a guardrail to ensure the maintenance of the refactored test behavior. We leverage MeteoR to validate the practical applicability of this approach across diverse test refactoring scenarios. This evaluation uses a catalog of common test refactorings that reflect real-world practices. The results indicate that MeteoR effectively detects changes in test behavior in most scenarios, demonstrating the efficacy of mutation testing in identifying issues during test code refactoring. However, the study also identifies limitations, particularly the occurrence of false negatives when refactorings modify how tests handle dependencies. These findings highlight the potential of the approach for verifying the maintenance of test behavior and contribute to the state-of-the-art by identifying limitations that can be addressed in future studies.

Wed 4 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change