TCSE logo 
 Sigsoft logo
Sustainability badge

This program is tentative and subject to change.

Sun 27 Apr 2025 11:00 - 11:22 at 206 - Flakiness in Specific/Previously Neglected Domains

Prior research has extensively studied flaky tests in various domains, such as web applications, mobile applica- tions, and Python projects in a range of multiple programming languages, including Java, Python, Android, JavaScript, Ruby, and more. However, little attention has been given to flaky tests in Rust—an emerging popular language known for its safety features relative to C/C++. Rust incorporates interesting features that make it easy to detect some flaky tests, e.g., the Rust standard library randomizes the order of elements in hash tables, effectively exposing implementation-dependent flakiness. However, Rust still has several sources of nondeterminism that can lead to flaky tests. We present our work-in-progress on studying flaky tests in Rust projects on GitHub. Searching through the closed GitHub issues and pull requests, we identified 1,146 issues potentially related to Rust flaky tests. We focus on flaky tests that are fixed, not just reported, as the fixes can offer valuable information on root causes, manifestation characteristics, and strategies of fixes. By far, we have inspected 53 tests. Our initial findings indicate that the predominant root causes include asynchronous wait (33.9%), concurrency issues (24.5%), logic errors (9.4%), and network-related problems (9.4%).

This program is tentative and subject to change.

Sun 27 Apr

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
Flakiness in Specific/Previously Neglected DomainsFTW at 206
11:00
22m
Paper
A Preliminary Study of Fixed Flaky Tests in Rust Projects on GitHub
FTW
Tom Schroeder University of Illinois Urbana-Champaign, Minh Phan University of Illinois Urbana-Champaign, Yang Chen University of Illinois at Urbana-Champaign
11:22
22m
Talk
Beyond Test Flakiness: A Manifesto for a Holistic Approach to Test Suite Health
FTW
Phil McMinn University of Sheffield, Muhammad Firhard Roslan University of Sheffield, Gregory Kapfhammer Allegheny College
11:45
22m
Paper
Empirically Evaluating Flaky Tests for Autonomous Driving Systems in Simulated Environments
FTW
Olek Osikowicz University of Sheffield, UK, Phil McMinn University of Sheffield, Donghwan Shin University of Sheffield
12:07
22m
Panel
Mini Panel 1
FTW

:
:
:
: