Tackling Flaky Tests: Understanding the Problem and Providing Practical Solutions
Wed 17 Nov 2021 20:12 - 20:14 at Kangaroo - LBR + DS Poster (1) (Wed 07:00 - 10:00) Chair(s): Maria Spichkova
Non-deterministically behaving tests impede software development as they hamper regression testing, destroy trust, and waste resources. This phenomenon, also called test flakiness, has received increasing attention over the past years. The multitude of both peer-reviewed literature and online blog articles touching the issue illustrates that flaky tests are deemed both a relevant research topic and a serious problem in everyday business. A major shortcoming of existing work aiming to mitigate flaky tests is its limited applicability since many of the proposed tools are highly relying on specific ecosystems. This issue also reflects on various attempts to investigate flaky tests: Using mostly similar sets of open-source Java projects, many studies are unable to generalize their findings to projects laying beyond this scope. On top of that, a holistic understanding of flaky tests also suffers from a lack of analyses focusing on the developers’ perspective with most existing studies taking a code-centric approach. With my work, I want to close these gaps: I plan to create an overarching and widely applicable framework that empowers developers to tackle flaky tests through existing and novel techniques and enables researchers to swiftly deploy and evaluate new approaches. As a starting point, I am studying test flakiness from previously unconsidered angles: I widen the scope of observation investigating flakiness beyond the realm of the Java ecosystem while also capturing the practitioners’ opinion. By adding to the understanding of the phenomenon I not only hope to close existing research gaps, but to retrieve a clear vision of how research on test flakiness can create value for developers working in the field.
Mon 15 NovDisplayed time zone: Hobart change
20:15 - 22:00 | |||
20:15 15mTalk | Gas Estimation and Optimization for Smart Contracts on Ethereum Doctoral Symposium Chunmiao Li National Institute of Informatics Pre-print | ||
20:30 15mTalk | Quality analysis of mobile applications with special focus on security aspects Doctoral Symposium Kristiina Rahkema University of Tartu Pre-print | ||
20:45 15mTalk | How can Subgroup Discovery help AIOps? Doctoral Symposium Youcef REMIL INSA Lyon, INFOLOGIC, Anes Bendimerad Infologic, Mehdi Kaytoue INFOLOGIC, Jean-François Boulicaut INSA Lyon Pre-print | ||
21:00 15mTalk | Semi-automated Cross-Component Issue Management and Impact Analysis Doctoral Symposium Sandro Speth Institute of Software Engineering, University of Stuttgart File Attached | ||
21:15 15mTalk | Tackling Flaky Tests: Understanding the Problem and Providing Practical Solutions Doctoral Symposium Martin Gruber BMW Group, University of Passau File Attached | ||
21:30 15mOther | Discussion with presenters Doctoral Symposium | ||
21:45 15mTalk | Closing of DS Doctoral Symposium |
Wed 17 NovDisplayed time zone: Hobart change
20:00 - 21:00 | LBR + DS Poster (1) (Wed 07:00 - 10:00)Doctoral Symposium / Late Breaking Results at Kangaroo Chair(s): Maria Spichkova RMIT University, Australia | ||
20:00 2mTalk | Time-Aware Coverage Criteria for Testing of AI-Enabled Hybrid Control Systems Late Breaking Results Zhenya Zhang Kyushu University, Deyun Lyu Kyushu university, Paolo Arcaini National Institute of Informatics
, Lei Ma University of Alberta, Ichiro Hasuo National Institute of Informatics, Japan, Jianjun Zhao Kyushu University | ||
20:02 2mTalk | Metamorphic Testing on Multi-module UAV Systems Late Breaking Results Rui Li Swinburne University of Technology, Huai Liu Swinburne University of Technology, Guannan Lou Macquarie University, Xi Zheng Macquarie University, Xiao Liu School of Information Technology, Deakin University, Tsong Yueh Chen Swinburne University of Technology | ||
20:04 2mTalk | Quality analysis of mobile applications with special focus on security aspects Doctoral Symposium Kristiina Rahkema University of Tartu Pre-print | ||
20:06 2mTalk | Semi-automated Cross-Component Issue Management and Impact Analysis Doctoral Symposium Sandro Speth Institute of Software Engineering, University of Stuttgart File Attached | ||
20:08 2mTalk | Cross-Lingual Transfer Learning Framework for Program Analysis Doctoral Symposium Zhiming Li Nanyang Technological University, Singapore | ||
20:10 2mTalk | Decoupling Server and Client Code Through Cloud-Native Domain-Specific Functions Late Breaking Results José Miguel Pérez-Álvarez NAVER LABS Europe, Adrian Mos NAVER LABS Europe, Benjamin V. Hanrahan Pennsylvania State University, Iyadunni J. Adenuga Pennsylvania State University | ||
20:12 2mTalk | Tackling Flaky Tests: Understanding the Problem and Providing Practical Solutions Doctoral Symposium Martin Gruber BMW Group, University of Passau File Attached | ||
20:14 2mTalk | Effectively Analyzing Evolving Software with Differential Facts Doctoral Symposium Xiuheng Wu Nanyang Technological University File Attached | ||
20:16 2mTalk | Automatically Annotating Sentences for Task-specific Bug Report Summarization Late Breaking Results Akalanka Galappaththi University of Alberta, John Anvik University of Lethbridge, Rafat Bin Islam University of Lethbridge | ||
20:18 2mTalk | Genetic Optimisation of C++ Applications Late Breaking Results Rafail Giavrimis Turing Intelligence Technology, Alexis Butler Turing Intelligence Technology, Constantin Cezar Petrescu University of Surrey, Michail Basios Turing Intelligence Technology, Santanu Dash University of Surrey, UK | ||
20:20 2mTalk | Revisiting Textual Feature of Bug-Triage Approach Late Breaking Results | ||
20:22 2mTalk | Training Automated Test Oracles to Identify Semantic Bugs Doctoral Symposium Charaka Geethal Monash University File Attached | ||
20:24 2mTalk | Automated Approach for System-level Testing of Unmanned Aerial Systems Doctoral Symposium Hassan Sartaj National University of Computer and Emerging Sciences, Islamabad, Pakistan File Attached | ||
20:26 2mTalk | How can Subgroup Discovery help AIOps? Doctoral Symposium Youcef REMIL INSA Lyon, INFOLOGIC, Anes Bendimerad Infologic, Mehdi Kaytoue INFOLOGIC, Jean-François Boulicaut INSA Lyon Pre-print | ||
20:28 2mTalk | Gas Estimation and Optimization for Smart Contracts on Ethereum Doctoral Symposium Chunmiao Li National Institute of Informatics Pre-print | ||
20:30 2mTalk | Fuzzing Methods Recommendation Based on Feature Vectors Doctoral Symposium Chi Zhang Jiangsu University File Attached |