Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia
Thu 18 May 2023 13:45 - 14:00 at Meeting Room 110 - Test quality and improvement Chair(s): Guowei Yang

Today’s software failures have two dominating root causes: code bugs and misconfigurations. To combat failure-inducing software changes, unified regression testing (URT) is needed to synergistically test the changed code and all changed production configurations (configs) for deployment reliability. However, URT could incur high cost to run a large number of tests under multiple configs. Regression test selection (RTS) can reduce regression testing cost. Unfortunately, no existing RTS technique reasons about code and config changes collectively. We introduce Unified Regression Testing Selection (uRTS) to effectively reduce the cost of URT. uRTS supports project changes on 1) code only, 2) configs only, and 3) both code and configs. It selects regular tests and configuration tests with a unified selection algorithm. The uRTS algorithm analyzes code and config dependencies of each test across runs and across configs. uRTS provides the same safety guarantee as the state-of-the-art RTS while selecting fewer tests and, more importantly, reducing the end-to-end testing time. We implemented uRTS on top of Ekstazi (a RTS tool for code changes) and Ctest (a config testing framework). We evaluate uRTS on hundreds of code revisions and dozens of configs of five large projects. The results show that uRTS reduces the end-to-end testing time, on average, by 3.64X compared to executing all tests and 1.87X compared to a competitive reference solution that directly extends RTS for URT.

Thu 18 May

Displayed time zone: Hobart change

13:45 - 15:15
Test quality and improvementTechnical Track / Journal-First Papers / DEMO - Demonstrations at Meeting Room 110
Chair(s): Guowei Yang University of Queensland
13:45
15m
Talk
Test Selection for Unified Regression Testing
Technical Track
Shuai Wang University of Illinois at Urbana-Champaign, Xinyu Lian University of Illinois at Urbana-Champaign, Darko Marinov University of Illinois at Urbana-Champaign, Tianyin Xu University of Illinois at Urbana-Champaign
Pre-print
14:00
15m
Talk
ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolutionary Search
Technical Track
Rongqi Pan University of Ottawa, Taher A Ghaleb University of Ottawa, Lionel Briand University of Luxembourg; University of Ottawa
14:15
15m
Talk
Measuring and Mitigating Gaps in Structural Testing
Technical Track
Soneya Binta Hossain University of Virginia, Matthew B Dwyer University of Virginia, Sebastian Elbaum University of Virginia, Anh Nguyen-Tuong University of Virginia
Pre-print
14:30
7m
Talk
FlaPy: Mining Flaky Python Tests at Scale
DEMO - Demonstrations
Martin Gruber BMW Group, University of Passau, Gordon Fraser University of Passau
Pre-print
14:37
7m
Talk
Scalable and Accurate Test Case Prioritization in Continuous Integration Contexts
Journal-First Papers
Ahmadreza Saboor Yaraghi University of Ottawa, Mojtaba Bagherzadeh University of Ottawa, Nafiseh Kahani University of Carlton, Lionel Briand University of Luxembourg; University of Ottawa
14:45
7m
Talk
Flakify: A Black-Box, Language Model-based Predictor for Flaky Tests
Journal-First Papers
Sakina Fatima University of Ottawa, Taher A Ghaleb University of Ottawa, Lionel Briand University of Luxembourg; University of Ottawa
14:52
7m
Talk
Developer-centric test amplification
Journal-First Papers
Carolin Brandt Delft University of Technology, Andy Zaidman Delft University of Technology
Pre-print
15:00
7m
Talk
How Developers Engineer Test Cases: An Observational Study
Journal-First Papers
Maurício Aniche Delft University of Technology, Christoph Treude University of Melbourne, Andy Zaidman Delft University of Technology
Pre-print