FalsifAI: Falsification of AI-Enabled Hybrid Control Systems Guided by Time-Aware Coverage Criteria
Modern Cyber-Physical Systems (CPSs) that need to perform complex control tasks (e.g., autonomous driving) are increasingly using AI-enabled controllers, mainly based on deep neural networks (DNNs). The quality assurance of such types of systems is of vital importance. However, their verification can be extremely challenging, due to their complexity and uninterpretable decision logic. Falsification is an established approach for CPS quality assurance, which, instead of attempting to prove the system correctness, aims at finding a time-variant input signal violating a formal specification describing the desired behavior; it often employs a search-based testing approach that tries to minimize the robustness of the specification, given by its quantitative semantics. However, guidance provided by robustness is mostly black-box and only related to the system output, but does not allow to understand whether the temporal internal behavior determined by multiple consecutive executions of the neural network controller has been explored sufficiently. To bridge this gap, in this paper, we make an early attempt at exploring the temporal behavior determined by the repeated executions of the neural network controllers in hybrid control systems and first propose eight time-aware coverage criteria specifically designed for neural network controllers in the context of CPS, which consider different features by design: the simple temporal activation of a neuron, the continuous activation of a neuron for a given duration, and the differential neuron activation behavior over time. Secondly, we introduce a falsification framework, named FalsifAI, that exploits the coverage information for better falsification guidance. Namely, inputs of the controller that increase the coverage (so improving the exploration of the DNN behaviors), are prioritized in the exploitation phase of robustness minimization. Our large-scale evaluation over a total of 3 typical CPS tasks, 6 system specifications, 18 DNN models and more than 12,000 experiment runs, demonstrates 1) the advantage of our proposed technique in outperforming two state-of-the-art falsification approaches, and 2) the usefulness of our proposed time-aware coverage criteria for effective falsification guidance.
Fri 19 MayDisplayed time zone: Hobart change
13:45 - 15:15 | Cyber-physical systems developmentSEIP - Software Engineering in Practice / Journal-First Papers / DEMO - Demonstrations at Meeting Room 102 Chair(s): Andrzej Wąsowski IT University of Copenhagen, Denmark | ||
13:45 15mTalk | Hybrid Cloudification of Legacy Software for Efficient Simulation of Gas Turbine Designs SEIP - Software Engineering in Practice Fozail Ahmad McGill University, Maruthi Rangappa , Neeraj Katiyar McGill University, Canada, Martin Staniszewski Siemens Energy, Daniel Varro Linköping University / McGill University | ||
14:00 15mTalk | Automated Misconfiguration Repair of Configurable Cyber-Physical Systems with Search: an Industrial Case Study on Elevator Dispatching Algorithms SEIP - Software Engineering in Practice Pre-print | ||
14:15 7mTalk | WirelessDT: A Digital Twin Platform for Real-Time Evaluation of Wireless Software Applications DEMO - Demonstrations Zhongzheng Lai The University of Sydney, Dong Yuan The University of Sydney, Huaming Chen The University of Sydney, Yu Zhang The University of Sydney, Wei Bao The University of Sydney Media Attached | ||
14:22 7mTalk | MROS: A framework for robot self-adaptation DEMO - Demonstrations Gustavo Rezende Silva Cognitive Robotics, Delft University of Technology, Darko Bozhinoski Université Libre de Bruxelles, Mario Garzon Oviedo Department of Cognitive Robotics, Delft University of Technology, Mariano Ramírez Montero Cognitive Robotics, Delft University of Technology, Nadia Hammoudeh Garcia Fraunhofer IPA, Harshavardhan Deshpande Fraunhofer IPA, Andrzej Wąsowski IT University of Copenhagen, Denmark, Carlos Hernández Corbato Delft University of Technology | ||
14:30 7mTalk | Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems Journal-First Papers Andrea Stocco Technical University of Munich & fortiss, Brian Pulfer University of Geneva, Paolo Tonella USI Lugano | ||
14:37 7mTalk | Uncertainty-aware Prediction Validator in Deep Learning Models for Cyber-physical System Data (Journal First Presentation) Journal-First Papers Ferhat Ozgur Catak University of Stavanger, Norway, Tao Yue Simula Research Laboratory, Shaukat Ali Simula Research Laboratory | ||
14:45 7mTalk | Uncertainty-aware Robustness Assessment of Industrial Elevator Systems Journal-First Papers Liping Han Nanjing University of Aeronautics and Astronautics & Simula Research Laboratory, Shaukat Ali Simula Research Laboratory, Tao Yue Simula Research Laboratory, Aitor Arrieta Mondragon University, Maite Arratibel Orona | ||
14:52 7mTalk | Learning Configurations of Operating Environment of Autonomous Vehicles to Maximize their Collisions Journal-First Papers Chengjie Lu Simula Research Laboratory and University of Oslo, Shi Yize Nanjing University of Aeronautics and Astronautics, Huihui Zhang Weifang University, Man Zhang Kristiania University, Tiexin Wang Nanjing University of Aeronautics and Astronautics, Tao Yue Simula Research Laboratory, Shaukat Ali Simula Research Laboratory Link to publication DOI Pre-print | ||
15:00 7mTalk | FalsifAI: Falsification of AI-Enabled Hybrid Control Systems Guided by Time-Aware Coverage Criteria Journal-First Papers Zhenya Zhang Kyushu University, Deyun Lyu Kyushu university, Paolo Arcaini National Institute of Informatics
, Lei Ma University of Alberta, Ichiro Hasuo National Institute of Informatics, Japan, Jianjun Zhao Kyushu University Link to publication DOI |