Shake Those System Parameters! On the Need for Parameter Coverage for Decision Systems
Decision systems such as Multiple-Criteria Decision Analysis systems formulate a decision process in terms of a mathematical function that takes into consideration different aspects of a problem. Testing such systems is crucial, as they are usually employed in safety-critical systems. A \emph{good} test suite for these systems should be able to exercise all the possible types of decisions that can be taken by the system. Classic structural coverage criteria do not provide good test suites in this sense, as they can be fulfilled by simple tests that only cover one possible type of decision. Therefore, in this paper we discuss the need for tailored coverage criteria for this class of systems, and we propose a criterion based on the perturbation of the decision systems’ parameters. We demonstrate the effectiveness of the criterion, compared to classic structural coverage criteria, on a path planner system for autonomous driving. We also discuss other benefits, such as the criterion helping explain \emph{why} a decision was made during a test.
Tue 16 NovDisplayed time zone: Hobart change
22:00 - 23:00 | ApplicationsResearch Papers / Industry Showcase / NIER track at Koala Chair(s): ingo Mueller Monash University | ||
22:00 20mTalk | Targeting Requirements Violations of Autonomous Driving Systems by Dynamic Evolutionary Search Research Papers Yixing Luo Peking University, Xiao-Yi Zhang National Institute of Informatics, Japan, Paolo Arcaini National Institute of Informatics
, Zhi Jin Peking University, Haiyan Zhao Peking University, Fuyuki Ishikawa National Institute of Informatics, Rongxin Wu Xiamen University, Tao Xie Peking University | ||
22:20 10mTalk | Evaluating Semantic Autocompletion of Business Processes with Domain Experts Industry Showcase | ||
22:30 10mTalk | Learning-based Assistant for Migrating Data of Enterprise Information Systems Industry Showcase Sayandeep Mitra Tata Consultancy Services, Debayan Mukherjee Tata Consultancy Services, Atreya Bandyopadhyay Tata Consultancy Services, Rajdip Chowdhury Tata Consultancy Services, Raveendra Kumar Medicherla Tata Consultancy Services, Indrajit Bhattacharya Tata Consultancy Services, Ravindra Naik TCS Research, TRDDC, India | ||
22:40 10mTalk | Shake Those System Parameters! On the Need for Parameter Coverage for Decision Systems NIER track Thomas Laurent Lero & University College Dublin, Paolo Arcaini National Institute of Informatics
, Fuyuki Ishikawa National Institute of Informatics, Anthony Ventresque University College Dublin |