Write a Blog >>
ICSE 2020
Wed 24 June - Thu 16 July 2020
Wed 8 Jul 2020 16:28 - 16:34 at Silla - A12-Testing Chair(s): Sasa Misailovic

Intermittent test failures (test flakiness) is common during continuous integration as modern software systems have become inherently non-deterministic. Understanding the root cause of test flakiness is crucial as intermittent test failures might be the result of real non-deterministic defects in the production code, rather than mere errors in the test code. Existing techniques for root causing test flakiness compare the runtime behaviour of passing and failing executions of the same test. They achieve this by repetitively executing a flaky test on an instrumented version of the system under test. This approach has two fundamental limitations: (i) code instrumentation might disrupt the normal behaviour of the test preventing the manifestation of test flakiness; (ii) passively re-executing a test many times could be insufficient for triggering intermittent test outcomes when test flakiness is rare or difficult to manifest. To address these limitations, we propose a new approach for root causing test flakiness that actively explores the non-deterministic space, without instrumenting code. Our novel idea is to repetitively execute a flaky test, under different execution clusters. Each cluster explores a certain non-deterministic dimension (e.g., concurrency, I/O, and networking) with dedicated software containers and fuzzy-driven resource load generators. The execution cluster that manifests the most balanced (or unbalanced) sets of passing and failing executions is likely to explain the broad type of test flakiness.

Wed 8 Jul

Displayed time zone: (UTC) Coordinated Universal Time change

16:05 - 17:05
A12-TestingJournal First / New Ideas and Emerging Results / Demonstrations / Technical Papers at Silla
Chair(s): Sasa Misailovic University of Illinois at Urbana-Champaign
16:05
12m
Talk
Practical Fault Detection in Puppet ProgramsArtifact ReusableTechnicalArtifact Available
Technical Papers
Thodoris Sotiropoulos Athens University of Economics and Business, Dimitris Mitropoulos Athens University of Economics and Business, Diomidis Spinellis Athens University of Economics and Business
16:17
8m
Talk
Empirical Assessment of Multimorphic TestingJ1
Journal First
Paul Temple PReCISE, NaDi, UNamur, Mathieu Acher (Univ Rennes, Inria, IRISA), Jean-Marc Jézéquel Univ Rennes - IRISA
16:25
3m
Talk
RTj: a Java framework for detecting and refactoring rotten green test casesDemo
Demonstrations
Matias Martinez Université Polytechnique Hauts-de-France, Anne Etien Université de Lille, CNRS, Inria, Centrale Lille, UMR 9189 –CRIStAL, Stéphane Ducasse INRIA Lille, Christopher Fuhrman École de technologie supérieure
Pre-print Media Attached
16:28
6m
Talk
A Container-Based Infrastructure for Fuzzy-Driven Root Causing of Flaky TestsNIER
New Ideas and Emerging Results
Valerio Terragni Università della Svizzera Italiana, Pasquale Salza University of Zurich, Filomena Ferrucci University of Salerno
Pre-print Media Attached
16:34
12m
Talk
Learning from, Understanding, and Supporting DevOps Artifacts for DockerArtifact ReusableTechnicalArtifact Available
Technical Papers
Jordan Henkel University of Wisconsin–Madison, Christian Bird Microsoft Research, Shuvendu K. Lahiri Microsoft Research, Thomas Reps University of Wisconsin-Madison, USA
16:46
8m
Talk
Improving Change Prediction Models with Code Smell-Related InformationJ1
Journal First
Gemma Catolino Delft University of Technology, Fabio Palomba University of Salerno, Francesca Arcelli Fontana University of Milano-Bicocca, Andrea De Lucia University of Salerno, Andy Zaidman TU Delft, Filomena Ferrucci University of Salerno
DOI Pre-print
16:54
3m
Talk
SMRL: A Metamorphic Security Testing Tool for Web SystemsDemo
Demonstrations
Phu X. Mai University of Luxembourg, Arda Goknil SnT, University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel C. Briand SnT Centre/University of Luxembourg