TARGET: Traffic Rule-Based Test Generation for Autonomous Driving via Validated LLM-Guided Knowledge Extraction
This program is tentative and subject to change.
Testing autonomous driving systems (ADSs) at scale requires systematic generation of diverse scenarios. Current practice depends on verbose DSLs such as OpenSCENARIO, where hundreds of lines of code are required to describe even simple manoeuvres. This process is error-prone, labor-intensive, and hinders large-scale testing.
TARGET addresses this challenge with an end-to-end framework that automatically generates ADS test scenarios directly from traffic rules. It introduces (i) a simplified compositional DSL for functional scenarios, (ii) an LLM-guided knowledge extraction pipeline with validation and syntax alignment to mitigate hallucination, and (iii) automated synthesis of simulator-executable test scripts.
We evaluated TARGET on 284 scenarios derived from 54 traffic rules across seven ADSs (including Autoware, Apollo, LAV, and MMFN) in CARLA, LGSVL, and MetaDrive. TARGET uncovered 610 violations, collisions, and timeouts, two of which were confirmed by ADS developers as genuine issues. The framework is fully open-sourced and has already solicited citations since publication.
Its influence is evident: the DSL and LLM-based knowledge extraction pipeline has been extended by the Purdue team in their FSE 2025 paper “Multi-modal Traffic Scenario Generation for Autonomous Driving System Testing”. In addition, formal methods researchers (e.g., the formal methods group at East China Normal University) are exploring DSL extensions as syntactic sugar for specification-driven verification of autonomous driving, while system security researchers (e.g., the Institute for Cybersecurity and Digital Trust at Ohio State University) are investigating its use for identifying vulnerabilities in autonomous driving systems. Presenting this paper in the Journal-First track at ASE 2025 will ensure the work reaches a broad audience, consolidates community uptake, and stimulates discussion across ASE’s core themes of software testing, program synthesis, DSLs, and AI/LLMs in SE, while further promoting its adoption in both academia and industry.
This program is tentative and subject to change.
Mon 17 NovDisplayed time zone: Seoul change
11:00 - 12:30 | |||
11:00 10mTalk | ADPerf: Investigating and Testing Performance in Autonomous Driving Systems Research Papers Tri Minh-Triet Pham Concordia University, Diego Elias Costa Concordia University, Canada, Weiyi Shang University of Waterloo, Jinqiu Yang Concordia University | ||
11:10 10mTalk | VRTestSniffer: Test Smell Detector for Virtual Reality (VR) Software Projects Research Papers Faraz Gurramkonda University of Michigan-Dearborn, Avishak Chakroborty University of Michigan-Dearborn, Bruce Maxim University of Michigan - Dearborn, Mohamed Wiem Mkaouer University of Michigan - Flint, Foyzul Hassan University of Michigan at Dearborn | ||
11:20 10mTalk | A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems Research Papers Stefano Carlo Lambertenghi Technische Universität München, fortiss GmbH, Mirena Flores Valdez Technical University of Munich, Andrea Stocco Technical University of Munich, fortiss Pre-print | ||
11:30 10mTalk | On the Robustness Evaluation of 3D Obstacle Detection Against Specifications in Autonomous Driving Research Papers Tri Minh-Triet Pham Concordia University, Bo Yang Concordia University, Jinqiu Yang Concordia University | ||
11:40 10mTalk | TARGET: Traffic Rule-Based Test Generation for Autonomous Driving via Validated LLM-Guided Knowledge Extraction Journal-First Track Yao Deng Macquarie University, Zhi Tu Purdue University, Jiaohong Yao Macquarie University, Mengshi Zhang TensorBlock, Tianyi Zhang Purdue University, Xi Zheng Macquarie University | ||
11:50 10mTalk | IMUFUZZER: Resilience-based Discovery of Signal Injection Attacks on Robotic Aerial Vehicles Research Papers Sudharssan Mohan University of Texas at Dallas, Kyeongseok Yang Korea University, Zelun Kong The University of Texas at Dallas, Yonghwi Kwon University of Maryland, Junghwan Rhee University of Central Oklahoma, Tyler Summers University of Texas at Dallas, Hongjun Choi DGIST, Heejo Lee Korea University, Chung Hwan Kim University of Texas at Dallas | ||
12:00 10mTalk | Argus: Resilience-Oriented Safety Assurance Framework for End-to-End ADSs Research Papers Dingji Wang Fudan University, You Lu Fudan University, Bihuan Chen Fudan University, Shuo Hao Fudan University, Haowen Jiang Fudan University, China, Yifan Tian Fudan University, Xin Peng Fudan University | ||
12:10 10mResearch paper | VRExplorer: A Model-based Approach for Automated Virtual Reality Scene Testing Research Papers Zhu Zhengyang Sun Yat-sen University, Hong-Ning Dai Hong Kong Baptist University, Hanyang Guo School of Software Engineering, Sun Yat-sen University, Zeqin Liao Sun Yat-sen University, Zibin Zheng Sun Yat-sen University Pre-print | ||
12:20 10mTalk | When Autonomous Vehicle Meets V2X Cooperative Perception: How Far Are We? Research Papers An Guo Nanjing University, Shuoxiao Zhang Nanjing University, Enyi Tang Nanjing University, Xinyu Gao , Haomin Pang Guangzhou University, Haoxiang Tian Nanyang Technological University, Singapore, Yanzhou Mu , Wu Wen Guangzhou University, Chunrong Fang Nanjing University, Zhenyu Chen Nanjing University Pre-print | ||