3rd International Flaky Tests Workshop 2026 (FTW 2026)FTW 2026

What is a flaky test?

Software developers rely on test cases to identify bugs in their code and to provide a signal as to their code’s correctness. Should such signals have a history of unreliability, they not only become less informative, but may also be considered untrustworthy. In the context of software testing, practitioners refer to these unreliable signals as flaky tests. The definition varies slightly, but a flaky test is generally defined as a test case that can pass and fail without changes to the test case code or the code under test.

Why are they such a big deal?

Concurrency and randomness are well-established causes among many others, though flakiness has far-reaching negative consequences regardless of origin. These consequences are felt by developers from small open-source projects to the likes of Google, Meta, and Microsoft. Flaky tests challenge the assumption that a test failure implies a bug, constituting a leading cause of “false alarm” test failures, and potentially more seriously, having the potential to mask the presence of a genuine bug. Flaky tests may lead to time wasted debugging spurious failures, leading developers to ignore future test failures. This is detrimental to software stability, because while a flaky test may be unreliable, it could still indicate a genuine bug in some instances. This is further exacerbated when flaky tests accumulate, as developers may lose trust in the entire test suite.

What are we going to do about it?

Flaky tests as a research topic has grown in interest significantly within the software engineering community in recent years. This has produced a wide array of empirical studies on the causes of flaky tests and experimental tools for their detection and repair. The International Flaky Test Workshop (FTW) will be held for the third consecutive time at ICSE 2026. The workshop welcomes submissions on topics relating to flaky tests and will provide an opportunity for academic researchers and industrial practitioners to exchange ideas about test flakiness. The workshop will feature a panel discussion with leading experts from both academia and industry. Please see the Call for Papers for more information.

Call for Papers

The primary objective of FTW is to foster collaboration and exchange between academia and industry. FTW welcomes submissions on topics relating to flaky tests and non-determinism in testing generally. The workshop will provide an opportunity for academic researchers and industrial practitioners to exchange ideas about test flakiness and to find out about current research directions and industrial challenges. The workshop is inclusive of quantitative, qualitative, and mixed-methods research. Topics of interest include (but are not limited to):

• Causes of flaky tests.
• Costs and consequences of flaky tests.
• Debugging of flaky tests.
• Detection of flaky tests.
• Mitigation of flaky tests.
• Non-determinism in testing generally.
• Repair of flaky tests.

We expect a significant portion of the day to be spent on presentations and discussions of extended abstracts, but there will also be more formal short paper presentations. Please note that due to ICSE restrictions, submissions cannot exceed 8 pages. Submissions can take one of two formats:

• Extended abstract (max. 2 pages including references): New ideas, problems and challenges, view points, work in progress. Extended abstracts are free of APC (article processing charge).
• Short paper (max. 6 pages + 2 pages references): Technical research, experience reports, empirical studies.

Submission

All submissions must be made via the following link: https://icse2026-ftw.hotcrp.com/

Each submission will be reviewed by the program committee with respect to suitability for the workshop, following a double-blind process for short papers and a single-blind process for extended abstracts. This means that the identity of short paper authors must not be revealed in their submissions. Please note that for this year’s submission, you are required to use the official “ACM Primary Article Template”. You can get this from the ACM Proceedings Template page. For those using LaTeX, make sure to use the sigconf option as well as the review option for line numbers to facilitate easy referencing by reviewers. As such, you can include the following LaTeX code at the beginning of your LaTeX document: \documentclass[sigconf,review]{acmart}.

More precise submission policies and formatting guidance can be found within the ICSE 2026 Research Track submission process.