Write a Blog >>
ISSTA 2020
Sat 18 - Wed 22 July 2020
Tue 21 Jul 2020 15:30 - 15:50 at Zoom - REGRESSION TESTING Chair(s): Alex Orso

Test-case prioritization (TCP) aims to detect regression bugs faster via reordering the tests run. While TCP has been studied for over 20 years, it was never evaluated with real test failures from open-source projects. In this work, we study the recent change-aware information retrieval (IR) technique for TCP. Prior work has shown it performing better than traditional coverage-based TCP techniques, but it was only evaluated on a small-scale dataset with a cost-unaware metric based on seeded faults/mutants. We extend the prior work by conducting a much larger and more realistic evaluation as well as proposing enhancements that substantially improve the performance. In particular, we evaluate the original technique on a large-scale, real-world software-evolution dataset with real failures using both cost-aware and cost-unaware metrics under various configurations. Also, we design and evaluate hybrid techniques combining the IR features, historical test execution time, and test failure frequencies. Our results show that the change-aware IR TCP technique outperforms state-of-the-art coverage-based techniques in this real-world setting, and our hybrid techniques improve even further upon the original IR technique. As a result, we explore various configurations of Moreover, we show that flaky tests have a substantial impact on evaluating the change-aware TCP techniques based on real test failures.

Tue 21 Jul
Times are displayed in time zone: (GMT-07:00) Tijuana, Baja California change

14:50 - 15:50: Technical Papers - REGRESSION TESTING at Zoom
Chair(s): Alex OrsoGeorgia Institute of Technology

Public Live Stream/Recording. Registered participants should join via the Zoom link distributed in Slack.

issta-2020-papers14:50 - 15:10
Wing LamUniversity of Illinois at Urbana-Champaign, August ShiThe University of Texas at Austin, Reed Oei, Sai ZhangGoogle Cloud, Michael D. ErnstUniversity of Washington, USA, Tao XiePeking University
DOI Media Attached
issta-2020-papers15:10 - 15:30
Patrice GodefroidMicrosoft Research, Daniel LehmannUniversity of Stuttgart, Marina PolishchukMicrosoft
DOI Media Attached
issta-2020-papers15:30 - 15:50
Qianyang Peng, August ShiThe University of Texas at Austin, Lingming ZhangThe University of Texas at Dallas