Write a Blog >>
Thu 13 Oct 2022 17:20 - 17:40 at Ballroom C East - Technical Session 29 - AI for SE II Chair(s): Tim Menzies

In presence of multiple objectives to be optimized in Search-Based Software Engineering (SBSE), Pareto search has been commonly adopted. It searches for a good approximation of the problem’s Pareto optimal solutions, from which the stakeholders choose the most preferred solution according to their preferences. However, when clear preferences of the stakeholders (e.g., a set of weights which reflect relative importance between objectives) are available prior to the search, weighted search is believed to be the first choice since it simplifies the search via converting the original multi-objective problem into a single-objective one and enable the search to focus on what only the stakeholders are interested in.

This paper questions such a “weighted search first” belief. We show that the weights can, in fact, be harmful to the search process even in the presence of clear preferences. Specifically, we conduct a large scale empirical study which consists of 38 systems/projects from three representative SBSE problems, together with two types of search budget and nine sets of weights, leading to 604 cases of comparisons. Our key finding is that weighted search reaches a certain level of solution quality by consuming relatively less resources at the early stage of the search; however, Pareto search is at the majority of the time (up to 77% of the cases) significantly better than its weighted counterpart, as long as we allow a sufficient, but not unrealistic search budget. This is a beneficial result, as it discovers a potentially new “rule-of-thumb” for the SBSE community: even when clear preferences are available, it is recommended to always consider Pareto search by default for multi-objective SBSE problems provided that solution quality is more important. Weighted search, in contrast, should only be preferred when the resource/search budget is limited, especially for expensive SBSE problems. This, together with other findings and actionable suggestions in the paper, allows us to codify pragmatic and comprehensive guidance on choosing weighted and Pareto search for SBSE under the circumstance that clear preferences are available. All code and data can be accessed at: https://github.com/ideas-labo/pareto-vs-weight-for-sbse.

Thu 13 Oct

Displayed time zone: Eastern Time (US & Canada) change

16:00 - 18:00
Technical Session 29 - AI for SE IIResearch Papers / Journal-first Papers at Ballroom C East
Chair(s): Tim Menzies North Carolina State University
16:00
20m
Research paper
Are Neural Bug Detectors Comparable to Software Developers on Variable Misuse Bugs?
Research Papers
Cedric Richter University of Oldenburg, Jan Haltermann University of Oldenburg, Marie-Christine Jakobs Technical University of Darmstadt, Felix Pauck Paderborn University, Germany, Stefan Schott Paderborn University, Heike Wehrheim University of Oldenburg
DOI Pre-print Media Attached File Attached
16:20
20m
Research paper
Learning Contract Invariants Using Reinforcement Learning
Research Papers
Junrui Liu University of California, Santa Barbara, Yanju Chen University of California at Santa Barbara, Bryan Tan Amazon Web Services, Işil Dillig University of Texas at Austin, Yu Feng University of California at Santa Barbara
16:40
20m
Research paper
Compressing Pre-trained Models of Code into 3 MB
Research Papers
Jieke SHI Singapore Management University, Zhou Yang Singapore Management University, Bowen Xu School of Information Systems, Singapore Management University, Hong Jin Kang Singapore Management University, Singapore, David Lo Singapore Management University
DOI Pre-print Media Attached
17:00
20m
Research paper
A Transferable Time Series Forecasting Service using Deep Transformer model for Online SystemsVirtual
Research Papers
Tao Huang Tencent, Pengfei Chen Sun Yat-Sen University, Jingrun Zhang School of Data and Computer Science, Sun Yat-sen University, Ruipeng Li Tencent, Rui Wang Tencent
17:20
20m
Paper
The Weights can be Harmful: Pareto Search versus Weighted Search in Multi-Objective Search-Based Software EngineeringVirtual
Journal-first Papers
Tao Chen Loughborough University, Miqing Li University of Birmingham
Pre-print
17:40
20m
Research paper
Robust Learning of Deep Predictive Models from Noisy and Imbalanced Software Engineering DatasetsVirtual
Research Papers
Zhong Li Nanjing, Minxue Pan Nanjing University, Yu Pei Hong Kong Polytechnic University, Tian Zhang Nanjing University, Linzhang Wang Nanjing University, Xuandong Li Nanjing University