Automating a Complete Software Test Process Using LLMs: An Automotive Case Study
Vehicle API testing verifies whether the interactions between a vehicle’s internal systems and external applications meet expectations, ensuring that users can access and control various vehicle functions and data. However, this task is inherently complex, requiring the alignment and coordination of API systems, communication protocols, and even vehicle simulation systems to develop valid test cases. In practical industrial scenarios, inconsistencies, ambiguities, and interdependencies across various documents and system specifications pose significant challenges. This paper presents a system designed for the automated testing of in-vehicle APIs. By clearly defining and segmenting the testing process, we enable Large Language Models (LLMs) to focus on specific tasks, ensuring a stable and controlled testing workflow. Experiments conducted on over 100 APIs demonstrate that our system effectively automates vehicle API testing. The results also confirm that LLMs can efficiently handle mundane tasks requiring human judgment, making them suitable for complete automation in similar industrial contexts.
Wed 30 AprDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | AutonomyResearch Track at 213 Chair(s): Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland | ||
11:00 15mTalk | A Differential Testing Framework to Identify Critical AV Failures Leveraging Arbitrary Inputs Research Track Trey Woodlief University of Virginia, Carl Hildebrandt University of Virginia, Sebastian Elbaum University of Virginia | ||
11:15 15mTalk | Automating a Complete Software Test Process Using LLMs: An Automotive Case Study Research Track Shuai Wang , Yinan Yu Chalmers University of Technology, Robert Feldt Chalmers | University of Gothenburg, Dhasarathy Parthasarathy Volvo Group Pre-print | ||
11:30 15mTalk | LLM-Agents Driven Automated Simulation Testing and Analysis of small Uncrewed Aerial Systems Research Track Venkata Sai Aswath Duvvuru Saint Louis University, Bohan Zhang Saint Louis University, Missouri, Michael Vierhauser University of Innsbruck, Ankit Agrawal Saint Louis University, Missouri Pre-print Media Attached | ||
11:45 15mTalk | Efficient Domain Augmentation for Autonomous Driving Testing Using Diffusion Models Research Track Luciano Baresi Politecnico di Milano, Davide Yi Xian Hu Politecnico di Milano, Andrea Stocco Technical University of Munich, fortiss, Paolo Tonella USI Lugano Pre-print | ||
12:00 15mTalk | GARL: Genetic Algorithm-Augmented Reinforcement Learning to Detect Violations in Marker-Based Autonomous Landing Systems Research Track Linfeng Liang Macquarie University, Yao Deng Macquarie University, Kye Morton Skyy Network, Valtteri Kallinen Skyy Network, Alice James Macquarie University, Avishkar Seth Macquarie University, Endrowednes Kuantama Macquarie University, Subhas Mukhopadhyay Macquarie University, Richard Han Macquarie University, Xi Zheng Macquarie University | ||
12:15 15mTalk | Decictor: Towards Evaluating the Robustness of Decision-Making in Autonomous Driving Systems Research Track Mingfei Cheng Singapore Management University, Xiaofei Xie Singapore Management University, Yuan Zhou Zhejiang Sci-Tech University, Junjie Wang Tianjin University, Guozhu Meng Institute of Information Engineering, Chinese Academy of Sciences, Kairui Yang DAMO Academy, Alibaba Group, China | ||