This talk explores the use of quality metrics to assess the performance of autonomous driving systems, with a focus on selecting metrics based on their correlation with human evaluations of driving behaviour. By leveraging mutation testing, these quality metrics are applied to generate oracles that identify faulty driving instances. The first part of the talk will provide an overview of mutation testing in deep learning (DL), discussing its general use and challenges. The focus will then shift to our study, in which we systematically analysed existing literature to select quality metrics, gathered human assessments of autonomous driving across various scenarios, identified the metrics most strongly correlated with human judgment, and used those metrics to generate oracles.

Bio: Gunel Jahangirova is a Lecturer (Assistant Professor) at King’s College London in the United Kingdom. Before joining KCL, she worked as a post-doctoral researcher at the Software Institute of Università della Svizzera Italiana (USI) in Lugano, Switzerland. She has her PhD in a joint program between Fondazione Bruno Kessler, Trento, Italy and University College London, London, UK. Her research interests include the test oracle problem, testing of AI systems, using LLMs for automated software testing and software engineering for quantum systems.