Write a Blog >>

Autonomous Driving Systems (ADSs) are safety-critical, and must be fully tested before being deployed on real-world roads. To comprehensively evaluate the performance of ADSs, it is essential to generate various safety-critical scenarios. Most of existing studies assess ADSs either by searching high dimensional input space, or using simple and pre-defined test scenarios, which are not efficient or not adequate. To better test ADSs, this paper proposes to automatically generate safety-critical test scenarios for ADSs by influential behavior patterns, which are mined from real traffic trajectories. Based on influential behavior patterns, a novel scenario generation technique, CRISCO, is presented to generate safety-critical scenarios for ADSs testing. CRISCO assigns participants to perform influential behavior patterns to challenge the ADS. It generates diverse test scenarios by solving a group of trajectory constraints, and improves the challenge of those non-critical scenarios by adding participants’ behavior from influential behavior patterns incrementally. We demonstrate CRISCO on an industrial-grade full-stack ADS platform, Baidu Apollo. The experiment results show that our approach can effectively and efficiently generate safety-critical scenarios to crash ADS, and it exposes 13 distinct types of safety violations in 12 hours. It also outperforms two state-of-art ADS testing techniques by exposing more 5 distinct types of safety violations on the same road.

Thu 13 Oct

Displayed time zone: Eastern Time (US & Canada) change

13:30 - 15:30
Technical Session 28 - Safety-Critical and Self-Adaptive SystemsIndustry Showcase / Tool Demonstrations / Research Papers / Late Breaking Results / NIER Track at Room 128
Chair(s): Eunsuk Kang Carnegie Mellon University
SAFA: A Tool for Supporting Safety Analysis in Evolving Software Systems
Tool Demonstrations
Alberto D. Rodriguez University of Notre Dame, Timothy Newman University of Notre Dame, Katherine R. Dearstyne University of Notre Dame, Jane Cleland-Huang University of Notre Dame
Research paper
Generating Critical Test Scenarios for Autonomous Driving Systems via Influential Behavior PatternsVirtual
Research Papers
Haoxiang Tian Institute of Software, Chinese Academy of Sciences, Guoquan Wu Institute of Software at Chinese Academy of Sciences, China, Jiren Yan Institute of Software, Chinese Academy of Sciences, Yan Jiang Institute of Software, Chinese Academy of Sciences, Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences, Wei Chen Institute of Software at Chinese Academy of Sciences, China, Shuo Li Institute of Software, Chinese Academy of Sciences, Dan Ye Institute of Software, Chinese Academy of Sciences
Research paper
Consistent Scene Graph Generation by Constraint OptimizationVirtual
Research Papers
Boqi Chen McGill University, Kristóf Marussy Budapest University of Technology and Economics, Sebastian Pilarski McGill University, Oszkár Semeráth Budapest University of Technology and Economics, Daniel Varro McGill University / Budapest University of Technology and Economics
Industry talk
A Drift Handling Approach for Self-Adaptive ML Software in Scalable Industrial ProcessesVirtual
Industry Showcase
Firas Bayram Department of Mathematics and Computer Science, Karlstad University, Sweden, Bestoun S. Ahmed Karlstad University Sweden, Erik Hallin Uddeholms AB, Sweden, Anton Engman Uddeholms AB, Sweden
SML4ADS: An Open DSML for Autonomous Driving Scenario Representation and GenerationVirtual
Late Breaking Results
Bo Li East China Normal University, Dehui Du East China Normal University, Sicong Chen East China Normal University, Minjun Wei East China Normal University, Chenghang Zheng East China Normal University, Xinyuan Zhang East China Normal University
Vision and Emerging Results
XSA: eXplainable Self-AdaptationVirtual
NIER Track
Matteo Camilli Free University of Bozen-Bolzano, Raffaela Mirandola Politecnico di Milano, Patrizia Scandurra University of Bergamo, Italy
File Attached
Industry talk
Design-Space Exploration for Decision-Support Software
Industry Showcase
Ate Penders Thales Research & Technology, Ana Lucia Varbanescu University of Twente, Gregor Pavlin Thales Research & Technology, Henk Sips Delft University of Technology