ASE 2024
Sun 27 October - Fri 1 November 2024 Sacramento, California, United States
Wed 30 Oct 2024 13:45 - 14:00 at Gardenia - Autonomous Systems Chair(s): Qingkai Shi

Deep Neural Networks (DNNs) for Autonomous Driving Systems (ADS) are typically trained on real-world images and tested using synthetic images from simulators. This approach results in training and test datasets with dissimilar distributions, which can potentially lead to erroneously decreased test accuracy. To address this issue, the deep learning literature suggests applying domain-to-domain translators to test datasets to bring them closer to the training datasets. However, translating images used for testing may unpredictably affect the reliability, effectiveness and efficiency of the testing process. Hence, this paper investigates the following questions in the context of ADS: Could translators reduce the effectiveness of images used for ADS-DNN testing and their ability to reveal faults in ADS-DNNs? Can translators result in excessive time overhead during simulation-based testing? To address these questions, we consider three domain-to-domain translators: CycleGAN and neural style transfer, both widely used in the literature, and SAEVAE, our proposed translator. Our results for two critical ADS tasks – lane keeping and object detection – indicate that translators significantly narrow the gap in ADS test accuracy caused by distribution dissimilarities between training and test data, with SAEVAE outperforming the other two translators. We show that, based on the recent diversity, coverage, and fault-revealing ability metrics for testing deep-learning systems, translators do not compromise the diversity and the coverage of test data nor do they lead to revealing fewer faults in ADS-DNNs. Furthermore, among the three translators considered, SAEVAE incurs a negligible overhead in simulation time and can be efficiently integrated into simulation-based testing. Finally, we observe that translators increase the correlation between offline and simulation-based testing results, which can help reduce the cost of simulation-based testing.

Wed 30 Oct

Displayed time zone: Pacific Time (US & Canada) change

13:30 - 15:00
Autonomous SystemsResearch Papers / Journal-first Papers / Industry Showcase at Gardenia
Chair(s): Qingkai Shi Nanjing University
13:30
15m
Talk
SoVAR: Build Generalizable Scenarios from Accident Reports for Autonomous Driving Testing
Research Papers
An Guo Nanjing University, Yuan Zhou Nanyang Technological University, Haoxiang Tian Nanyang Technological University, Chunrong Fang Nanjing University, Yunjian Sun Nanjing University, Weisong Sun Nanyang Technological University, Xinyu Gao , Luu Anh Tuan Nanyang Technological University, Yang Liu Nanyang Technological University, Zhenyu Chen Nanjing University
Pre-print
13:45
15m
Talk
Bridging the Gap between Real-world and Synthetic Images for Testing Autonomous Driving Systems
Research Papers
Mohammad Hossein Amini University of Ottawa, Shiva Nejati University of Ottawa
14:00
15m
Talk
In-Simulation Testing of Deep Learning Vision Models in Autonomous Robotic Manipulators
Industry Showcase
Dmytro Humeniuk Polytechnique Montréal, Houssem Ben Braiek Sycodal, Thomas Reid Sycodal, Foutse Khomh Polytechnique Montréal
14:15
15m
Talk
LeGEND: A Top-Down Approach to Scenario Generation of Autonomous Driving Systems Assisted by Large Language ModelsACM SigSoft Distinguished Paper Award
Research Papers
Shuncheng Tang University of Science and Technology of China, Zhenya Zhang Kyushu University, Japan, Jixiang Zhou University of Science and Technology of China, Lei Wang National University of Defense Technology, Yuan Zhou Zhejiang Sci-Tech University, Yinxing Xue University of Science and Technology of China
14:30
15m
Talk
ROCAS: Root Cause Analysis of Autonomous Driving Accidents via Cyber-Physical Co-mutationACM SigSoft Distinguished Paper Award
Research Papers
Shiwei Feng Purdue University, Yapeng Ye Purdue University, Qingkai Shi Nanjing University, Zhiyuan Cheng Purdue University, Xiangzhe Xu Purdue University, Siyuan Cheng Purdue University, Hongjun Choi DGIST, Xiangyu Zhang Purdue University
14:45
15m
Talk
The IDEA of Us: An Identity-Aware Architecture for Autonomous Systems
Journal-first Papers
Carlos Gavidia-Calderon The Alan Turing Institute, Anastasia Kordoni Lancaster University (UK), Amel Bennaceur The Open University, UK, Mark Levine Lancaster University, Bashar Nuseibeh The Open University, UK; Lero, University of Limerick, Ireland