Test-Case Prioritization for Configuration TestingACM SIGSOFT Distinguished Paper
Sat 17 Jul 2021 02:10 - 02:30 at ISSTA 1 - Session 21 (time band 2) Testing 3 Chair(s): Rohan Padhye
Configuration changes are among the dominant causes of failures of large-scale software system deployment. Given the velocity of configuration changes, typically at the scale of hundreds to thousands of times daily in modern cloud systems, checking these configuration changes is critical to prevent failures due to misconfigurations. Recent work has proposed configuration testing, Ctest, a technique that tests configuration changes together with the code that uses the changed configurations. Ctest can automatically generate a large number of ctests that can effectively detect misconfigurations, including those that are hard to detect by traditional techniques. However, running ctests can take a long time to detect misconfigurations. Inspired by traditional test-case prioritization (TCP) that aims to reorder test executions to speed up detection of regression code faults, we propose to apply TCP to reorder ctests to speed up detection of misconfigurations. We extensively evaluate a total of 84 traditional and novel ctest-specific TCP techniques. The experimental results on five widely used cloud projects demonstrate that TCP can substantially speed up misconfiguration detection. Our study provides guidelines for applying TCP to configuration testing in practice.
Thu 15 JulDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
19:00 - 20:20 | Session 12 (time band 1) Configuration and Dependency Technical Papers at ISSTA 2 Chair(s): Jonathan Bell Northeastern University | ||
19:00 20mTalk | Fixing Dependency Errors for Python Build Reproducibility Technical Papers Suchita Mukherjee University of California at Davis, Abigail Almanza University of California at Davis, Cindy Rubio-González University of California at Davis DOI | ||
19:20 20mTalk | Test-Case Prioritization for Configuration TestingACM SIGSOFT Distinguished Paper Technical Papers Runxiang Cheng University of Illinois at Urbana-Champaign, Lingming Zhang University of Illinois at Urbana-Champaign, Darko Marinov University of Illinois at Urbana-Champaign, Tianyin Xu University of Illinois at Urbana-Champaign DOI | ||
19:40 20mTalk | The Impact of Tool Configuration Spaces on the Evaluation of Configurable Taint Analysis for Android Technical Papers DOI Pre-print | ||
20:00 20mTalk | Challenges and Opportunities: An In-Depth Empirical Study on Configuration Error Injection Testing Technical Papers Wang Li National University of Defense Technology, Zhouyang Jia National University of Defense Technology, Shanshan Li National University of Defense Technology, Yuanliang Zhang National University of Defense Technology, Teng Wang National University of Defense Technology, Erci Xu National University of Defense Technology, Ji Wang National University of Defense Technology, Liao Xiangke National University of Defense Technology DOI File Attached |
Sat 17 JulDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
01:10 - 02:30 | Session 21 (time band 2) Testing 3Technical Papers at ISSTA 1 Chair(s): Rohan Padhye Carnegie Mellon University | ||
01:10 20mTalk | Continuous Test Suite Failure Prediction Technical Papers DOI Media Attached | ||
01:30 20mTalk | Toward Optimal MC/DC Test Case Generation Technical Papers Sangharatna Godboley National Institute of Technology Warangal, Joxan Jaffar National University of Singapore, Rasool Maghareh Huawei, Arpita Dutta National University of Singapore DOI | ||
01:50 20mTalk | Challenges and Opportunities: An In-Depth Empirical Study on Configuration Error Injection Testing Technical Papers Wang Li National University of Defense Technology, Zhouyang Jia National University of Defense Technology, Shanshan Li National University of Defense Technology, Yuanliang Zhang National University of Defense Technology, Teng Wang National University of Defense Technology, Erci Xu National University of Defense Technology, Ji Wang National University of Defense Technology, Liao Xiangke National University of Defense Technology DOI File Attached | ||
02:10 20mTalk | Test-Case Prioritization for Configuration TestingACM SIGSOFT Distinguished Paper Technical Papers Runxiang Cheng University of Illinois at Urbana-Champaign, Lingming Zhang University of Illinois at Urbana-Champaign, Darko Marinov University of Illinois at Urbana-Champaign, Tianyin Xu University of Illinois at Urbana-Champaign DOI |