Testing CPS with Design Assumptions-Based Metamorphic Relations and Genetic Programming
Cyber-Physical Systems (CPSs) software is used to enforce desired behaviours on physical systems. To test the interaction between the CPS software and the system’s physics, engineers provide traces of desired physical states and observe traces of the actual physical states. CPS requirements describe how closely the actual physical traces should track the desired traces. These requirements are typically defined for specific, simple input traces such as step or ramp sequences, and thus are not applicable to arbitrary inputs. This limits the availability of oracles for CPSs. Our recent work proposes an approach to testing CPSs using control-theoretical design assumptions instead of requirements. This approach circumvents the oracle problem by leveraging the control-theoretical guarantees that are provided when the design assumptions are satisfied. To address the test case generation and oracle problems, researchers have proposed metamorphic testing, which is based on the study of relations across tests, i.e., metamorphic relations (MRs). In this work, we define MRs based on the design assumptions and explore combinations of these MRs using genetic programming to generate CPS test cases. This enables the generation of CPS input traces with potentially arbitrary shapes, together with associated expected output traces. We use the deviation from the expected output traces to guide the generation of input traces that falsify the MRs. Our experiment results show that the MR-falsification provides engineers with new information, helping them identify passed and failed test cases. Furthermore, we show that the generation of traces that falsify the MRs is a non-trivial problem, which cannot be addressed with a random generation approach but is successfully addressed by our approach based on genetic search.
Wed 15 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | Testing and Analysis 2Journal-first Papers / SE In Practice (SEIP) / Research Track at Oceania II Chair(s): Andrea Stocco Technical University of Munich, fortiss | ||
11:00 15mTalk | An industrial experience report on applying search-based boundary input generation to cyber-physical systems Journal-first Papers Pablo Valle Mondragon University, Vincenzo Riccio University of Udine, Aitor Arrieta Mondragon University, Paolo Tonella USI Lugano, Maite Arratibel Orona | ||
11:15 15mTalk | Testing CPS with Design Assumptions-Based Metamorphic Relations and Genetic Programming Journal-first Papers Claudio Mandrioli University of Luxembourg, Seung Yeob Shin University of Luxembourg, Domenico Bianculli University of Luxembourg, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland Link to publication DOI Pre-print | ||
11:30 15mTalk | Signal Feature Coverage and Testing for CPS Dataflow Models Journal-first Papers Ezio Bartocci TU Wien, Leonardo Mariani University of Milano-Bicocca, Dejan Nickovic Austrian Institute of Technology, Drishti Yadav University of Luxembourg, Luxembourg | ||
11:45 15mTalk | Uncovering Failures in Cyber-Physical System State Transitions: A Fuzzing-Based Approach Applied to sUAS Research Track Theodore Chambers University of Notre Dame, Arturo Miguel Russell Bernal University of Notre Dame, Michael Vierhauser University of Innsbruck, Jane Cleland-Huang University of Notre Dame Pre-print | ||
12:00 15mTalk | Vision Language Model-based Testing of Industrial Autonomous Mobile Robots SE In Practice (SEIP) Jiahui Wu Simula Research Laboratory and University of Oslo, Chengjie Lu Simula Research Laboratory and University of Oslo, Aitor Arrieta Mondragon University, Shaukat Ali Simula Research Laboratory and Oslo Metropolitan University, Thomas Peyrucain PAL Robotics | ||
12:15 15mTalk | Misbehavior Forecasting for Focused Autonomous Driving Systems Testing Research Track M M Abid Naziri North Carolina State University, Stefano Carlo Lambertenghi Technische Universität München, fortiss GmbH, Andrea Stocco Technical University of Munich, fortiss, Marcelo d'Amorim North Carolina State University Pre-print | ||