SSBSE 2022
Thu 17 - Fri 18 November 2022 Singapore
co-located with ESEC/FSE 2022
Fri 18 Nov 2022 14:30 - 15:30 at Virtual 3 (Whova) - Tutorial Chair(s): Jeongju Sohn

Search-Based Software Engineering (SBSE) has been becoming an increasingly important research paradigm for automating and solving different software engineering tasks. When the considered tasks have more than one objective/criterion to be optimised, they are called multi-objective ones. In such a scenario, the outcome is typically a set of incomparable solutions (i.e., being Pareto nondominated to each other), and then a common question faced by many SBSE practitioners is: how to evaluate the obtained sets by using the right methods and indicators in the SBSE context? In this tutorial, we seek to provide a systematic methodology and guideline for answering this question. We start off by discussing why we need formal evaluation methods/indicators for multi-objective optimisation problems in general, and the result of a survey on how they have been dominantly used in SBSE. This is then followed by a detailed introduction of representative evaluation methods and quality indicators used in SBSE, including their behaviors and preferences. In the meantime, we demonstrate the patterns and examples of potentially misleading usages/choices of evaluation methods and quality indicators from the SBSE community, highlighting their consequences. Afterwards, we present a systematic methodology that can guide the selection and use of evaluation methods and quality indicators for a given SBSE problem in general. Lastly, we showcase a real-world multi-objective SBSE case study, in which we demonstrate the consequences of incorrect use of evaluation methods/indicators and exemplify the implementation of the guidance provided.

Fri 18 Nov

Displayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change

14:00 - 15:30
TutorialTutorial at Virtual 3 (Whova)
Chair(s): Jeongju Sohn University of Luxembourg, Luxembourg
Methodology and Guidelines for Evaluating Multi-Objective Search-Based Software Engineering
Miqing Li University of Birmingham, Tao Chen Loughborough University
Link to publication Pre-print Media Attached File Attached