Write a Blog >>
ISSTA 2021
Sun 11 - Sat 17 July 2021 Online
co-located with ECOOP and ISSTA 2021
Fri 16 Jul 2021 08:00 - 08:20 at ISSTA 1 - Session 15 (time band 3) Testing 1 Chair(s): Phil McMinn
Fri 16 Jul 2021 19:20 - 19:40 at ISSTA 1 - Session 19 (time band 1) Testing 2 Chair(s): Rohan Padhye

Regression test selection (RTS) and prioritization (RTP) techniques aim to reduce testing efforts and developer feedback time after a change to the code base. Using various information sources, including test traces, build dependencies, version control data, and test histories, they have been shown to be effective. However, not all of these sources are guaranteed to be available and accessible for arbitrary continuous integration (CI) environments. In contrast, metadata from version control systems (VCSs) and CI systems are readily available and inexpensive. Yet, corresponding RTP and RTS techniques are scattered across research and often only evaluated on synthetic faults or in a specific industrial context. It is cumbersome for practitioners to identify insights that apply to their context, let alone to calibrate associated parameters for maximum cost-effectiveness. This paper consolidates existing work on RTP and unsafe RTS into an actionable methodology to build and evaluate such approaches that exclusively rely on CI and VCS metadata. To investigate how these approaches from prior research compare in heterogeneous settings, we apply the methodology in a large-scale empirical study on a set of 23 projects covering 37,000 CI logs and 76,000 VCS commits. We find that these approaches significantly outperform established RTP baselines and, while still triggering 90% of the failures, we show that practitioners can expect to save on average 84% of test execution time for unsafe RTS. We also find that it can be beneficial to limit training data, features from test history work better than change-based features, and, somewhat surprisingly, simple and well-known heuristics often outperform complex machine-learned models.

Fri 16 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

08:00 - 09:00
Session 15 (time band 3) Testing 1Technical Papers at ISSTA 1
Chair(s): Phil McMinn University of Sheffield
08:00
20m
Talk
Empirically Evaluating Readily Available Information for Regression Test Optimization in Continuous Integration
Technical Papers
Daniel Elsner TU Munich, Florian Hauer TU Munich, Alexander Pretschner TU Munich, Silke Reimer IVU Traffic Technologies
DOI
08:20
20m
Talk
Toward Optimal MC/DC Test Case Generation
Technical Papers
Sangharatna Godboley National Institute of Technology Warangal, Joxan Jaffar National University of Singapore, Rasool Maghareh Huawei, Arpita Dutta National University of Singapore
DOI
08:40
20m
Talk
Log-Based Slicing for System-Level Test Cases
Technical Papers
Salma Messaoudi University of Luxembourg, Donghwan Shin University of Luxembourg, Annibale Panichella Delft University of Technology; University of Luxembourg, Domenico Bianculli University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa
DOI Media Attached
18:20 - 20:00
Session 19 (time band 1) Testing 2Technical Papers at ISSTA 1
Chair(s): Rohan Padhye Carnegie Mellon University
18:20
20m
Talk
Model-Based Testing of Networked Applications
Technical Papers
Yishuai Li University of Pennsylvania, Benjamin C. Pierce University of Pennsylvania, Steve Zdancewic University of Pennsylvania
DOI Pre-print
18:40
20m
Talk
Validating Static Warnings via Testing Code Fragments
Technical Papers
Ashwin Kallingal Joshy Iowa State University, Xueyuan Chen Iowa State University, Benjamin Steenhoek Iowa State University, Wei Le Iowa State University
DOI
19:00
20m
Talk
Gramatron: Effective Grammar-Aware Fuzzing
Technical Papers
Prashast Srivastava Purdue University, Mathias Payer EPFL
DOI Pre-print Media Attached File Attached
19:20
20m
Talk
Empirically Evaluating Readily Available Information for Regression Test Optimization in Continuous Integration
Technical Papers
Daniel Elsner TU Munich, Florian Hauer TU Munich, Alexander Pretschner TU Munich, Silke Reimer IVU Traffic Technologies
DOI
19:40
20m
Talk
Log-Based Slicing for System-Level Test Cases
Technical Papers
Salma Messaoudi University of Luxembourg, Donghwan Shin University of Luxembourg, Annibale Panichella Delft University of Technology; University of Luxembourg, Domenico Bianculli University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa
DOI Media Attached