ICST 2026
Mon 18 - Fri 22 May 2026 Daejeon, South Korea
Tue 19 May 2026 14:00 - 14:25 at Room 101 - Test Flakiness Chair(s): Phil McMinn

Test flakiness is a significant issue in industry, affecting test efficiency and product quality. While extensive research has examined the impact of flaky tests, many root causes remain unexplored, particularly in the context of dynamic languages such as JavaScript. In this paper, we conduct a systematic evaluation of the impact of environmental factors on test flakiness in JavaScript. We first executed test suites across multiple environmental configurations to determine whether changes in the environment could lead to flaky behavior. We selected three environmental factors to manipulate: the operating system, the Node.js version, and the browser. We identified a total of 65 environmental flaky projects, with 28 related to operating system issues, five to Node.js version compatibility, 16 to a combination of operating system and Node.js issues, and 17 related to browser compatibility. To address environmental flakiness, we developed a lightweight mitigation approach, js- env-sanitizer, that can sanitize environmental-related flaky tests by skipping and reporting them (rather than failing), allowing CI builds to continue/succeed without rerunning entire test suites. The tool achieves high accuracy with minimal performance or configuration overhead, and currently supports three popular JavaScript testing frameworks (Jest, Mocha, and Vitest).

Tue 19 May

Displayed time zone: Seoul change

14:00 - 15:30
Test FlakinessResearch Papers / Short Papers, Vision and Emerging Results at Room 101
Chair(s): Phil McMinn University of Sheffield
14:00
25m
Talk
A Systematic Evaluation of Environmental Flakiness in JavaScript TestsArtifact ReviewedArtifact Available
Research Papers
Negar Hashemi Massey University, Amjed Tahir Massey University, August Shi The University of Texas at Austin, Shawn Rasheed UCOL, Rachel Blagojevic Massey University
14:25
25m
Talk
Pinpointing Flakiness in Web Tests via DOM Event Analysis and LLM-based Reasoning
Research Papers
Yu Pei University of Luxembourg, Jeongju Sohn Kyungpook National University, Sarra Habchi Cohere, Canada, Mike Papadakis University of Luxembourg
14:50
25m
Talk
Automatically Resolving Mismatched Invocations on Test Doubles in Test SuitesArtifact ReviewedArtifact Available
Research Papers
Mengzhen Li University of Minnesota, Mattia Fazzini University of Minnesota
15:15
15m
Talk
Towards an Actionable Flakiness Score
Short Papers, Vision and Emerging Results
Joanna Kisaakye University of Antwerp, Mutlu Beyazıt University of Antwerp and Flanders Make vzw, Serge Demeyer University of Antwerp and Flanders Make vzw