Failures or False Alarms? Validating Tests and Failures for Cyber Physical Systems
Software testing is about detecting failures, but not every failure necessarily indicates a genuine fault in the system under test. Some failures are spurious – caused by invalid test inputs or by flaky test outputs. While these spurious failures can arise in many contexts, they are especially prevalent in deep learning–enabled and cyber-physical systems, which often operate autonomously in complex, unpredictable environments. In such systems, virtually any environmental factor can act as an input, and the system’s internal decision-making – driven by deep learning models – may be nondeterministic or difficult to interpret. In this talk, I will discuss three solutions to ensure the validity of test inputs and increase the robustness of test outputs: (1) Generating interpretable constraints that characterize valid test inputs, (2) Developing effort-optimized, human-assisted methods for validating test inputs, and (3) Employing generative models to improve the consistency of test results across different simulation environments. I will illustrate these solutions through case studies from cyber-physical systems, autonomous driving, and network systems.
Sat 3 MayDisplayed time zone: Eastern Time (US & Canada) change
09:00 - 10:30 | Opening and Keynote SessionDeepTest at 213 Chair(s): Jinhan Kim Università della Svizzera italiana (USI) | ||
09:00 10mDay opening | Opening DeepTest | ||
09:10 80mKeynote | Failures or False Alarms? Validating Tests and Failures for Cyber Physical Systems DeepTest Shiva Nejati University of Ottawa |