Applications of Search-based Software Testing to Trustworthy Artificial Intelligence

Lionel C. Briand / University of Ottawa & University of Luxembourg

Abstract

Increasingly, many systems, including critical ones, rely on machine learning (ML) components to achieve autonomy or adaptiveness. Such components, having no specifications or source code, impact the way we develop but also verify such systems. This talk will report on experiences and lessons learned in applying search-based solutions to test and analyse such ML-enabled systems. Indeed, our results have shown that metaheuristic search plays a key role in enabling the effective test automation of ML models and the systems they are integrated into. Though other techniques are also required to achieve scalability and enable safety analysis, for example, the black-box nature of ML components naturally lends itself to search-based solutions.

Bio

Lionel C. Briand is a professor of software engineering and has shared appointments between (1) The University of Ottawa, Canada and (2) The SnT centre for Security, Reliability, and Trust, University of Luxembourg. In collaboration with colleagues, over 25 years, he has run many collaborative research projects with companies in the automotive, satellite, aerospace, energy, financial, and legal domains. Lionel was elevated to the grades of IEEE Fellow and ACM Fellow for his work on software testing and verification. He was granted the IEEE Computer Society Harlan Mills award, the ACM SIGSOFT outstanding research award, and the IEEE Reliability Society engineer-of-the-year award, respectively in 2012, 2022, and 2013. He received an ERC Advanced grant in 2016 — on the topic of modelling and testing cyber-physical systems — which is the most prestigious individual research award in the European Union. He currently holds a Canada Research Chair (Tier 1) on “Intelligent Software Dependability and Compliance”. His research interests include: software testing and verification, applications of AI in software engineering, model-driven software development, requirements engineering, and empirical software engineering.

Genetic Improvement of Software

Justyna Petke / University College London

Abstract

Genetic improvement uses computational search to improve existing software with respect to a user-defined objective function, while retaining some existing behaviour, usually captured by testing. Work on genetic improvement has already resulted in several awards. GI has been used, for instance, to automate the process of program repair, to speed up software for a particular domain, and to minimize memory and energy consumption. GI has also been used to transplant functionality from one software to another in an automated way. I will give an overview of the genetic improvement area and present key components of a GI framework.

Bio

Justyna Petke is a Principal Research Fellow and a Proleptic Associate Professor at the Centre for Research on Evolution, Search and Testing (CREST), located in the Department of Computer Science, University College London, UK. She is also a member of the Software Optimisation, Learning and Analytics Research (SOLAR) group at UCL. Her research focuses on Genetic Improvement, in particular use of search approaches to optimise software’s various properties, such as runtime and energy consumption, as well as to fix bugs and transplant new functionality. Her work on GI was awarded, among others, two SSBSE Challenge Track prizes, and two `Humies’, awarded for human-competitive results. She holds an EPSRC Early Career Fellowship on Automated Software Specialisation Using Genetic Improvement. Justyna was PC co-Chair of SSBSE 2017 and serves on the Editorial Board for the ASE, EMSE, GPEM and EAAI journals.