ICST 2023
Sun 16 - Thu 20 April 2023 Dublin, Ireland
Mon 17 Apr 2023 16:00 - 16:20 at Hanover - Session 7: Testing for Safery & Security Chair(s): Eric Bodden

An input to a system reveals a non-robust behaviour when, by making a small change in the input, the output of the system changes from acceptable (passing) to unacceptable (failing) or vice versa. Identifying inputs that lead to non-robust behaviours is important for many types of systems, e.g., cyber-physical and network systems, whose inputs are prone to perturbations. In this paper, we propose an approach that combines simulation-based testing with regression tree models to generate value ranges for inputs in response to which a system is likely to exhibit non-robust behaviours. We apply our approach to a network traffic-shaping system (NTSS) – a novel case study from the network domain. In this case study, developed and conducted in collaboration with a network solutions provider, RabbitRun Technologies, input ranges that lead to non-robustness are of interest as a way to identify and mitigate network quality-of-service issues. We demonstrate that our approach accurately characterizes non-robust test inputs of NTSS by achieving a precision of 84% and a recall of 100%, significantly outperforming a standard baseline. In addition, we show that there is no statistically significant difference between the results obtained from our simulated testbed and a hardware testbed with identical configurations. Finally we describe lessons learned from our industrial collaboration, offering insights about how simulation helps discover unknown and undocumented behaviours as well as a new perspective on using non-robustness as a measure for system re-configuration.

Mon 17 Apr

Displayed time zone: Dublin change

16:00 - 18:00
Session 7: Testing for Safery & Security Industry / Research Papers / Journal-First Papers / Previous Editions at Hanover
Chair(s): Eric Bodden Heinz Nixdorf Institut, Paderborn University and Fraunhofer IEM
Learning Non-robustness using Simulation-based Testing: a Network Traffic-shaping Case Study
Baharin Aliashrafi Jodat University of Ottawa, Shiva Nejati University of Ottawa, Mehrdad Sabetzadeh University of Ottawa, Patricio Saavedra RabbitRun Technologies Inc
Test environments for large-scale software systems – an industrial study of intrinsic and extrinsic success factors
Journal-First Papers
Torvald Mårtensson Saab AB, Göran Ancher Saab AB, Daniel Ståhl Ericsson AB
Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs
Previous Editions
Shaukat Ali Simula Research Laboratory, Paolo Arcaini National Institute of Informatics , Xinyi Wang , Tao Yue Simula Research Laboratory
Heap Fuzzing: Automatic Garbage Collection Testing with Directed Random Events
Research Papers
Guillermo Polito Inria, Cristal, UMR 9189, Université de Lille, Pablo Tesone Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRIStAL, Pharo Consortium, Jean Privat Université du Québec à Montréal (UQAM), Nahuel Palumbo Université Lille, CNRS, Centrale Lille, Inria, UMR 9189 - CRIStAL, Stéphane Ducasse Inria; University of Lille; CNRS; Centrale Lille; CRIStAL
MagicMirror: Towards High-Coverage Fuzzing of Smart Contracts
Research Papers
Huadong Feng University of Texas at Arlington, Xiaolei Ren University of Texas at Arlington, Qiping Wei University of Texas at Arlington, Jeff Yu Lei University of Texas at Arlington, Raghu Kacker National Institute of Standards and Technology, Richard Kuhn National Institute of Standards and Technology, Dimitris Simos SBA Research