Write a Blog >>
ISSTA 2020
Sat 18 - Wed 22 July 2020
Tue 21 Jul 2020 15:30 - 15:50 at Zoom - REGRESSION TESTING Chair(s): Alex Orso

Test-case prioritization (TCP) aims to detect regression bugs faster via reordering the tests run. While TCP has been studied for over 20 years, it was never evaluated with real test failures from open-source projects. In this work, we study the recent change-aware information retrieval (IR) technique for TCP. Prior work has shown it performing better than traditional coverage-based TCP techniques, but it was only evaluated on a small-scale dataset with a cost-unaware metric based on seeded faults/mutants. We extend the prior work by conducting a much larger and more realistic evaluation as well as proposing enhancements that substantially improve the performance. In particular, we evaluate the original technique on a large-scale, real-world software-evolution dataset with real failures using both cost-aware and cost-unaware metrics under various configurations. Also, we design and evaluate hybrid techniques combining the IR features, historical test execution time, and test failure frequencies. Our results show that the change-aware IR TCP technique outperforms state-of-the-art coverage-based techniques in this real-world setting, and our hybrid techniques improve even further upon the original IR technique. As a result, we explore various configurations of Moreover, we show that flaky tests have a substantial impact on evaluating the change-aware TCP techniques based on real test failures.

Tue 21 Jul

Displayed time zone: Tijuana, Baja California change

14:50 - 15:50
REGRESSION TESTINGTechnical Papers at Zoom
Chair(s): Alex Orso Georgia Institute of Technology

Public Live Stream/Recording. Registered participants should join via the Zoom link distributed in Slack.

14:50
20m
Talk
Dependent-Test-Aware Regression Testing Techniques
Technical Papers
Wing Lam University of Illinois at Urbana-Champaign, August Shi The University of Texas at Austin, Reed Oei , Sai Zhang Google Cloud, Michael D. Ernst University of Washington, USA, Tao Xie Peking University
DOI Media Attached
15:10
20m
Talk
Differential Regression Testing for REST APIs
Technical Papers
Patrice Godefroid Microsoft Research, Daniel Lehmann University of Stuttgart, Marina Polishchuk Microsoft
DOI Media Attached
15:30
20m
Talk
Empirically Revisiting and Enhancing IR-Based Test-Case Prioritization
Technical Papers
Qianyang Peng , August Shi The University of Texas at Austin, Lingming Zhang The University of Texas at Dallas
DOI