Write a Blog >>
ASE 2021
Sun 14 - Sat 20 November 2021 Australia
Wed 17 Nov 2021 08:50 - 08:55 at Kangaroo - Bugs I Chair(s): Elena Sherman
Wed 17 Nov 2021 10:00 - 10:02 at Kangaroo - Tool Demo (2) Chair(s): Mattia Fazzini

A test case that intermittently passes or fails when performed under the same version of source code and test code is said to be flaky. The presence of flaky tests wastes testing time and effort. The most popular approach in industry to detect flakiness is ReRun. The idea behind ReRun is very simple: failing test cases are re-executed many times looking for inconsistencies in the output. Despite its simplicity, the ReRun strategy is very expensive both in terms of time and in terms of computational resources. This is particularly true for contexts where thousands of test cases are performed on a daily basis. Reducing the rerunning overhead is, thus, of utmost importance. This paper presents SHAKER, an open-source tool for detecting flakiness in time-constrained tests by adding noise in the execution environment. The main idea behind SHAKER is to add stressing tasks that compete with the test execution for the use of resources (CPU or memory). SHAKER is available as a GitHub Actions workflow that can be seamlessly integrated with any GitHub project. Alternatively, SHAKER can also be used via its provided Command Line Interface. In our evaluation, SHAKER was able to discover more flaky tests than ReRun and in a faster way (less re-executions); besides, our approach revealed tens of new flaky tests that went undetected by ReRun even after 50 re-executions. Thanks to its flexibility and ease of use, we believe that SHAKER can be useful for both practitioners and researchers.

Wed 17 Nov

Displayed time zone: Hobart change

08:00 - 09:00
Bugs IResearch Papers / Industry Showcase / Tool Demonstrations at Kangaroo
Chair(s): Elena Sherman Boise State University
08:00
20m
Research paper
On the Real-World Effectiveness of Static Bug Detectors at Finding Null Pointer Exceptions
Research Papers
David A Tomassi University of California, Davis, Cindy Rubio-González University of California at Davis
08:20
20m
Talk
Subtle Bugs Everywhere: Generating Documentation for Data Wrangling Code
Research Papers
Chenyang Yang Peking University, Shurui Zhou University of Toronto, Jin L.C. Guo McGill University, Christian Kästner Carnegie Mellon University
08:40
10m
Talk
Reducing Time-To-Fix For Fuzzer Bugs
Industry Showcase
Rui Abreu Faculty of Engineering, University of Porto, Portugal, Franjo Ivančić Google, Filip Niksic Google, Hadi Ravanbakhsh Google, Ramesh Viswanathan Google
08:50
5m
Talk
Shaker: a Tool for Detecting More Flaky Tests Faster
Tool Demonstrations
Marcello Cordeiro Federal University of Pernambuco, Denini Silva Federal University of Pernambuco, Leopoldo Teixeira Federal University of Pernambuco, Breno Miranda Federal University of Pernambuco, Marcelo d'Amorim Federal University of Pernambuco
Link to publication
10:00 - 11:00
Tool Demo (2)Tool Demonstrations at Kangaroo
Chair(s): Mattia Fazzini University of Minnesota
10:00
2m
Talk
Shaker: a Tool for Detecting More Flaky Tests Faster
Tool Demonstrations
Marcello Cordeiro Federal University of Pernambuco, Denini Silva Federal University of Pernambuco, Leopoldo Teixeira Federal University of Pernambuco, Breno Miranda Federal University of Pernambuco, Marcelo d'Amorim Federal University of Pernambuco
Link to publication
10:04
2m
Talk
GenTree: Inferring Configuration Interactions using Decision Trees
Tool Demonstrations
KimHao Nguyen University of Nebraska-Lincoln, ThanhVu Nguyen George Mason University
10:06
2m
Talk
DEVIATE: A Deep Learning Variance Testing Framework
Tool Demonstrations
Hung Viet Pham University of Waterloo, Mijung Kim Purdue University, Lin Tan Purdue University, Yaoliang Yu University of Waterloo, Nachiappan Nagappan Microsoft Research
10:08
2m
Talk
FLACK: Localizing Faults in Alloy Models
Tool Demonstrations
Guolong Zheng University of Nebraska Lincoln, ThanhVu Nguyen George Mason University, Simón Gutiérrez Brida University of Rio Cuarto and CONICET, Argentina, Germán Regis Universidad Nacional de Río Cuarto, Marcelo F. Frias Dept. of Software Engineering Instituto Tecnológico de Buenos Aires, Nazareno Aguirre University of Rio Cuarto and CONICET, Argentina, Hamid Bagheri University of Nebraska-Lincoln
10:10
2m
Talk
Scalable Fuzzing of Program Binaries with E9AFL
Tool Demonstrations
Xiang Gao National University of Singapore, Gregory J. Duck National University of Singapore, Abhik Roychoudhury National University of Singapore