Write a Blog >>
MET 2021
Wed 2 Jun 2021
co-located with ICSE 2021
Wed 2 Jun 2021 10:20 - 10:40 at MET Room - Session 1: Autonomous Driving and Simulation Chair(s): Pak-Lok Poon

Deep Neural Network based models are widely used for perception and control in autonomous driving. Recent work leverages metamorphic testing to improve defect detection but is limited to using only an equality-based metamorphic relation. Thus, it does not provide expressiveness for users to define custom metamorphic relations nor means to automatically generate meaningful inputs based on such expressive metamorphic relations that reflect real-world traffic behaviors.

In this paper, we preliminary design and evaluate BMT, a declarative behavior driven development-based metamorphic testing framework.BMT enables domain experts to specify custom traffic behaviors—a car shall decelerate by x% when a bicycle is in front,etc. It then automatically translates a human-written behavior to a corresponding metamorphic relation and synthesizes meaningful test inputs using a variety of image and graphics processing techniques.

Our preliminary evaluation shows that BMT can detect a significant number of erroneous predictions of three driving models for speed predictions. These detected erroneous predictions are manually examined and confirmed by six human judges as meaningful traffic violations. By automating test generation from custom behaviors,BMT enables experts to easily express domain-specific constraints and finds meaningful violations of such constraints.

Wed 2 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:20 - 11:00
Session 1: Autonomous Driving and SimulationMET 2021 at MET Room
Chair(s): Pak-Lok Poon School of Engineering & Technology, Central Queensland University, Australia
BMT: Behavior Driven Development-based MetamorphicTesting for Autonomous Driving Models
MET 2021
Yao Deng Macquarie University, Guannan Lou Macquarie University, Xi Zheng Macquarie University, Tianyi Zhang Harvard University, USA, Miryung Kim University of California at Los Angeles, USA, Huai Liu Swinburne University of Technology, Chen Wang CSIRO DATA61, Tsong Yueh Chen Swinburne University of Technology
Media Attached
Enhancing Euro NCAP Standards with Metamorphic Testing for Verification of Advanced Driver-Assistance Systems
MET 2021
Muhammad Iqbal University of Wollongong, Jia Cheng Han University of Wollongong, Zhi Quan (George) Zhou University of Wollongong, Australia, Dave Towey University of Nottingham Ningbo China
Media Attached

Information for Participants
Wed 2 Jun 2021 10:20 - 11:00 at MET Room - Session 1: Autonomous Driving and Simulation Chair(s): Pak-Lok Poon
Info for room MET Room:

Go directly to this room on Clowdr