ICST 2026
Mon 18 - Fri 22 May 2026 Daejeon, South Korea
Tue 19 May 2026 14:25 - 14:50 at Room 101 - Test Flakiness Chair(s): Phil McMinn

Flaky tests in web applications frequently arise from nondeterministic interactions between asynchronous events and dynamic DOM updates. Traditional debugging techniques, including fault localization, struggle to diagnose such failures due to the complex, event-driven, and continuously evolving nature of modern web interfaces. This paper introduces DoeFL, a novel approach that combines structured analysis of DOM event sequences with large language model (LLM) reasoning to identify the root causes of web flakiness. DoeFL models temporal causal relationships between DOM events and flaky test failures using Lamport logical clocks. It then uses LLMs to capture semantic inconsistencies and intent mismatches often missed by purely structural analyses. We evaluate DoeFL on 122 real-world flaky test cases collected from 47 web projects. Using only its Lamport-based structural component, DoeFL achieves 47.5% Top-1 and 82.0% Top-5 localization accuracy. When augmented with LLM-based reasoning, performance further improves, achieving Top-1 accuracy between 65.6% and 77.9% and Top-5 accuracy between 89.3% and 95.1%,= depending on the selected LLM.

Tue 19 May

Displayed time zone: Seoul change

14:00 - 15:30
Test FlakinessResearch Papers / Short Papers, Vision and Emerging Results at Room 101
Chair(s): Phil McMinn University of Sheffield
14:00
25m
Talk
A Systematic Evaluation of Environmental Flakiness in JavaScript TestsArtifact ReviewedArtifact Available
Research Papers
Negar Hashemi Massey University, Amjed Tahir Massey University, August Shi The University of Texas at Austin, Shawn Rasheed UCOL, Rachel Blagojevic Massey University
14:25
25m
Talk
Pinpointing Flakiness in Web Tests via DOM Event Analysis and LLM-based Reasoning
Research Papers
Yu Pei University of Luxembourg, Jeongju Sohn Kyungpook National University, Sarra Habchi Cohere, Canada, Mike Papadakis University of Luxembourg
14:50
25m
Talk
Automatically Resolving Mismatched Invocations on Test Doubles in Test SuitesArtifact ReviewedArtifact Available
Research Papers
Mengzhen Li University of Minnesota, Mattia Fazzini University of Minnesota
15:15
15m
Talk
Towards an Actionable Flakiness Score
Short Papers, Vision and Emerging Results
Joanna Kisaakye University of Antwerp, Mutlu Beyazıt University of Antwerp and Flanders Make vzw, Serge Demeyer University of Antwerp and Flanders Make vzw