Test Case Prioritization for Acceptance Testing of Cyber Physical Systems: A Multi-Objective Search-Based Approach
Acceptance testing validates that a system meets its requirements and determines whether it can be sufficiently trusted and put into operation. For cyber physical systems (CPS), acceptance testing is a hardware-in-the-loop process conducted in a (near-)operational environment. Acceptance testing of a CPS often necessitates that the test cases be prioritized, as there are usually too many scenarios to consider given time constraints. CPS acceptance testing is further complicated by the uncertainty in the environment and the impact of testing on hardware. We propose an automated test case prioritization approach for CPS acceptance testing, accounting for time budget constraints, uncertainty, and hardware damage risks. Our approach is based on multi-objective search, combined with a test case minimization algorithm that eliminates redundant operations from an ordered sequence of test cases. We evaluate our approach on a representative case study from the satellite domain. The results indicate that, compared to test cases that are prioritized manually by satellite engineers, our automated approach more than doubles the number of test cases that fit into a given time frame, while reducing to less than one third the number of operations that entail the risk of damage to key hardware components.
Mon 16 JulDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
14:00 - 15:30 | Testing and Fault LocalizationISSTA Technical Papers at Zurich II Chair(s): Cindy Rubio-González University of California, Davis | ||
14:00 20mTalk | Test Case Prioritization for Acceptance Testing of Cyber Physical Systems: A Multi-Objective Search-Based Approach ISSTA Technical Papers Seung Yeob Shin SnT Centre/University of Luxembourg, Shiva Nejati SnT Centre/University of Luxembourg, Mehrdad Sabetzadeh SnT Centre/University of Luxembourg, Lionel C. Briand SnT Centre/University of Luxembourg, Frank Zimmer SES Techcom | ||
14:20 20mTalk | Bench4BL: Reproducibility Study on the Performance of IR-Based Bug Localization ISSTA Technical Papers Jaekwon Lee University of Luxembourg, Luxembourg, Dongsun Kim University of Luxembourg, Tegawendé F. Bissyandé University of Luxembourg, Luxembourg, Woosung Jung Seoul National University of Education, Yves Le Traon University of Luxembourg | ||
14:40 20mTalk | Automated Test Mapping and Coverage for Network Topologies ISSTA Technical Papers Per Erik Strandberg Westermo Research and Development AB, Thomas Ostrand , Elaine Weyuker Mälardalen University, Daniel Sundmark Mälardalen University, Wasif Afzal Mälardalen University | ||
15:00 20mTalk | Evaluating Test-Suite Reduction in Real-World Software Evolution ISSTA Technical Papers August Shi University of Illinois at Urbana-Champaign, Alex Gyori Facebook, Muhammad Suleman Mahmood University of Illinois at Urbana-Champaign, Peiyuan Zhao University of Illinois at Urbana-Champaign, Darko Marinov University of Illinois at Urbana-Champaign | ||
15:20 10m | Q&A in groups ISSTA Technical Papers |