Wed 17 Nov 2021 10:00 - 10:02 at Kangaroo - Tool Demo (2) Chair(s): Mattia Fazzini
A test case that intermittently passes or fails when performed under the same version of source code and test code is said to be flaky. The presence of flaky tests wastes testing time and effort. The most popular approach in industry to detect flakiness is ReRun. The idea behind ReRun is very simple: failing test cases are re-executed many times looking for inconsistencies in the output. Despite its simplicity, the ReRun strategy is very expensive both in terms of time and in terms of computational resources. This is particularly true for contexts where thousands of test cases are performed on a daily basis. Reducing the rerunning overhead is, thus, of utmost importance. This paper presents SHAKER, an open-source tool for detecting flakiness in time-constrained tests by adding noise in the execution environment. The main idea behind SHAKER is to add stressing tasks that compete with the test execution for the use of resources (CPU or memory). SHAKER is available as a GitHub Actions workflow that can be seamlessly integrated with any GitHub project. Alternatively, SHAKER can also be used via its provided Command Line Interface. In our evaluation, SHAKER was able to discover more flaky tests than ReRun and in a faster way (less re-executions); besides, our approach revealed tens of new flaky tests that went undetected by ReRun even after 50 re-executions. Thanks to its flexibility and ease of use, we believe that SHAKER can be useful for both practitioners and researchers.
Wed 17 NovDisplayed time zone: Hobart change
08:00 - 09:00 | Bugs IResearch Papers / Industry Showcase / Tool Demonstrations at Kangaroo Chair(s): Elena Sherman Boise State University | ||
08:00 20mResearch paper | On the Real-World Effectiveness of Static Bug Detectors at Finding Null Pointer Exceptions Research Papers David A Tomassi University of California, Davis, Cindy Rubio-González University of California at Davis | ||
08:20 20mTalk | Subtle Bugs Everywhere: Generating Documentation for Data Wrangling Code Research Papers Chenyang Yang Peking University, Shurui Zhou University of Toronto, Jin L.C. Guo McGill University, Christian Kästner Carnegie Mellon University | ||
08:40 10mTalk | Reducing Time-To-Fix For Fuzzer Bugs Industry Showcase Rui Abreu Faculty of Engineering, University of Porto, Portugal, Franjo Ivančić Google, Filip Niksic Google, Hadi Ravanbakhsh Google, Ramesh Viswanathan Google | ||
08:50 5mTalk | Shaker: a Tool for Detecting More Flaky Tests Faster Tool Demonstrations Marcello Cordeiro Federal University of Pernambuco, Denini Silva Federal University of Pernambuco, Leopoldo Teixeira Federal University of Pernambuco, Breno Miranda Federal University of Pernambuco, Marcelo d'Amorim Federal University of Pernambuco Link to publication |
10:00 - 11:00 | |||
10:00 2mTalk | Shaker: a Tool for Detecting More Flaky Tests Faster Tool Demonstrations Marcello Cordeiro Federal University of Pernambuco, Denini Silva Federal University of Pernambuco, Leopoldo Teixeira Federal University of Pernambuco, Breno Miranda Federal University of Pernambuco, Marcelo d'Amorim Federal University of Pernambuco Link to publication | ||
10:04 2mTalk | GenTree: Inferring Configuration Interactions using Decision Trees Tool Demonstrations | ||
10:06 2mTalk | DEVIATE: A Deep Learning Variance Testing Framework Tool Demonstrations Hung Viet Pham University of Waterloo, Mijung Kim Purdue University, Lin Tan Purdue University, Yaoliang Yu University of Waterloo, Nachiappan Nagappan Microsoft Research | ||
10:08 2mTalk | FLACK: Localizing Faults in Alloy Models Tool Demonstrations Guolong Zheng University of Nebraska Lincoln, ThanhVu Nguyen George Mason University, Simón Gutiérrez Brida University of Rio Cuarto and CONICET, Argentina, Germán Regis Universidad Nacional de Río Cuarto, Marcelo F. Frias Dept. of Software Engineering Instituto Tecnológico de Buenos Aires, Nazareno Aguirre University of Rio Cuarto and CONICET, Argentina, Hamid Bagheri University of Nebraska-Lincoln | ||
10:10 2mTalk | Scalable Fuzzing of Program Binaries with E9AFL Tool Demonstrations Xiang Gao National University of Singapore, Gregory J. Duck National University of Singapore, Abhik Roychoudhury National University of Singapore |