Tackling Flaky Tests: Understanding the Problem and Providing Practical Solutions
Non-deterministically behaving tests impede software development as they hamper regression testing, destroy trust, and waste resources. This phenomenon, also called test flakiness, has received increasing attention over the past years. The multitude of both peer-reviewed literature and online blog articles touching the issue illustrates that flaky tests are deemed both a relevant research topic and a serious problem in everyday business. A major shortcoming of existing work aiming to mitigate flaky tests is its limited applicability since many of the proposed tools are highly relying on specific ecosystems. This issue also reflects on various attempts to investigate flaky tests: Using mostly similar sets of open-source Java projects, many studies are unable to generalize their findings to projects laying beyond this scope. On top of that, a holistic understanding of flaky tests also suffers from a lack of analyses focusing on the developers’ perspective with most existing studies taking a code-centric approach. With my work, I want to close these gaps: I plan to create an overarching and widely applicable framework that empowers developers to tackle flaky tests through existing and novel techniques and enables researchers to swiftly deploy and evaluate new approaches. As a starting point, I am studying test flakiness from previously unconsidered angles: I widen the scope of observation investigating flakiness beyond the realm of the Java ecosystem while also capturing the practitioners’ opinion. By adding to the understanding of the phenomenon I not only hope to close existing research gaps, but to retrieve a clear vision of how research on test flakiness can create value for developers working in the field.
Mon 15 NovDisplayed time zone: Hobart change
20:15 - 22:00
|Gas Estimation and Optimization for Smart Contracts on Ethereum
Chunmiao Li National Institute of InformaticsPre-print
|Quality analysis of mobile applications with special focus on security aspects
Kristiina Rahkema University of TartuPre-print
|How can Subgroup Discovery help AIOps?
Youcef REMIL INSA Lyon, INFOLOGIC, Anes Bendimerad Infologic, Mehdi Kaytoue INFOLOGIC, Jean-François Boulicaut INSA LyonPre-print
|Semi-automated Cross-Component Issue Management and Impact Analysis
Sandro Speth Institute of Software Engineering, University of StuttgartFile Attached
|Tackling Flaky Tests: Understanding the Problem and Providing Practical Solutions
Martin Gruber BMW Group, University of PassauFile Attached
|Discussion with presenters
|Closing of DS