ICSE 2025 (series) / FTW 2025 (series) / 2nd International Flaky Tests Workshop 2025 (FTW 2025) /
Beyond Test Flakiness: A Manifesto for a Holistic Approach to Test Suite Health
Sun 27 Apr 2025 11:22 - 11:45 at 206 - Flakiness in Specific/Previously Neglected Domains
Large numbers of flaky tests are a sure sign of a dysfunctional, or “unhealthy”, test suite. In this paper, we identify several further indicators of unhealthy test suites, arguing that trade-offs exist among some indicators, with others complementing one another. We encourage researchers and practitioners not to stop at test flakiness—or any individual metric of “good” or “bad” tests—and instead focus on developing and adopting techniques and tools that holistically address test suite health.
Sun 27 AprDisplayed time zone: Eastern Time (US & Canada) change
Sun 27 Apr
Displayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | |||
11:00 22mPaper | A Preliminary Study of Fixed Flaky Tests in Rust Projects on GitHub FTW Tom Schroeder University of Illinois Urbana-Champaign, Minh Phan University of Illinois Urbana-Champaign, Yang Chen University of Illinois at Urbana-Champaign | ||
11:22 22mTalk | Beyond Test Flakiness: A Manifesto for a Holistic Approach to Test Suite Health FTW Phil McMinn University of Sheffield, Muhammad Firhard Roslan University of Bristol, Gregory Kapfhammer Allegheny College | ||
11:45 22mPaper | Empirically Evaluating Flaky Tests for Autonomous Driving Systems in Simulated Environments FTW Olek Osikowicz University of Sheffield, UK, Phil McMinn University of Sheffield, Donghwan Shin The University of Sheffield Pre-print | ||
12:07 22mPanel | Mini Panel 1 FTW |