Regression testing is a testing activity that aims to ensure that existing functionality is preserved when introducing changes. The goal of regression test selection (RTS) is to reduce the cost of regression testing by only re-executing tests that are affected by changes. Lately, research on RTS has focused on the languages Java and C++. Despite Rust being an increasingly relevant systems programming language, there are no RTS tools available for this language so far.
In this paper, we present and evaluate RustyRTS, the first RTS technique and tool for Rust. It provides both module-level and function-level RTS. Its function-level variants can rely on either static or dynamic code analysis to select affected tests. We evaluate RustyRTS in an empirical study in terms of safety, precision, and effectiveness. When applied to changes resulting from mutation testing on 9 open-source projects, RustyRTS selected 99.99%, 97.87% and 99.97% of all tests that failed in consequence of a modification in code using a module-level, static and dynamic RTS approach respectively. For cases of unsafe behavior, i.e., tests that have not been selected although failing due to such changes, we find plausible explanations. By applying RustyRTS to changes from the Git history of 13 repositories on GitHub, we effectively reduce the average end-to-end (e2e) testing time on the majority of projects. Our results show that the two function-level approaches outperform the coarser module-level one, especially on longer-running test suites. On average, the e2e testing time has been reduced to 67.80%, 62.52% and 52.79% of retest-all by module-level, static and dynamic RustyRTS respectively. Lastly, we provide a novel solution for dynamic dispatch and compile-time function evaluation, two contexts that impose a special challenge on approaches to RTS.
Wed 2 AprDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
16:00 - 17:00 | Test Case Selection, Prioritisation, ReductionResearch Papers / Industry at Aula Magna (AM) Chair(s): Andrea Stocco Technical University of Munich, fortiss | ||
16:00 15mTalk | The Impact of List Reduction for Language Agnostic Test Case Reducers Research Papers Tobias Heineken Friedrich-Alexander-Universität Erlangen-Nürnberg, Michael Philippsen Friedrich-Alexander Universität Erlangen-Nürnberg (FAU), Programming Systems Group | ||
16:15 15mTalk | RustyRTS: Regression Test Selection for Rust Research Papers Simon Hundsdorfer Technical University of Munich, Roland Würsching Technical University of Munich, Alexander Pretschner TU Munich | ||
16:30 15mTalk | ML-Based Test Case Prioritization: A Research and Production Perspective in CI Environments Industry Md Asif Khan Ontario Tech University, Akramul Azim Ontario Tech University, Ramiro Liscano Ontario Tech University, Kevin Smith International Business Machines Corporation (IBM), Yee-Kang Chang International Business Machines Corporation (IBM), Gkerta Seferi International Business Machines Corporation (IBM), Qasim Tauseef International Business Machines Corporation (IBM) | ||
16:45 15mTalk | Evaluating Machine Learning-Based Test Case Prioritization in the Real World: An Experiment with SAP HANA Industry |