Is It the Best Solution? Testing An Optimisation Algorithm with Metamorphic Testing
Optimisation algorithms play a vital role in solving complex real-world problems by iteratively comparing various solutions to find the optimal or the best solution. However, testing them poses challenges due to their “non-testable” nature, where a reliable test oracle is lacking. Traditional testing techniques may not directly address whether these algorithms yield the best solution. In this context, Metamorphic Testing (MT) emerges as a promising approach. MT leverages Metamorphic Relations (MRs) to indirectly test the System Under Test (SUT) by examining input-output pairs and revealing inconsistencies based on MRs. In this paper, we apply the MT approach to a black-box industrial optimisation algorithm and present our observations and findings. We identify successful aspects, challenges, and opportunities for further research. The findings from our study are expected to shed light on the practical feasibility of MT for testing optimisation algorithms. The paper provides a formal definition of MT, an overview of related work in optimisation algorithms, and details of the industrial context, methodology, and results.
Wed 13 DecDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:00 - 12:20 | Software Testing and Quality AssuranceResearch Papers / Organization / Short Papers and Posters / Industry Papers at W211 Chair(s): Dietmar Pfahl University of Tartu | ||
11:00 10mResearch paper | An Experience in the Evaluation of Fault Prediction Research Papers Luigi Lavazza Università degli Studi dell'Insubria, Sandro Morasca Università degli Studi dell'Insubria, Gabriele Rotoloni | ||
11:10 10mIndustry talk | Is It the Best Solution? Testing An Optimisation Algorithm with Metamorphic Testing Industry Papers Alejandra Duque-Torres University of Tartu, Claus Klammer Software Competence Center Hagenberg, Stefan Fischer Software Competence Center Hagenberg, Dietmar Pfahl University of Tartu | ||
11:20 10mShort-paper | Impacts of Program Structures on Code Coverage of Generated Test Suites Short Papers and Posters | ||
11:30 10mResearch paper | Anomaly Detection Through Container Testing: A Survey of Company Practices Research Papers Salla Timonen University of Jyväskylä, Maha Sroor University of Jyväskylä, Rahul Mohanani University of Jyväskylä, Tommi Mikkonen University of Jyvaskyla | ||
11:40 10mShort-paper | The Effects of Soft Assertion on Spectrum-based Fault Localization Short Papers and Posters Kouhei Mihara Osaka University, Shinsuke Matsumoto Osaka University, Shinji Kusumoto Osaka University | ||
11:50 10mIndustry talk | Characterizing Requirements Smells Industry Papers | ||
12:00 10mResearch paper | Do Exceptional Behavior Tests Matter on Spectrum-based Fault Localization? Research Papers Haruka Yoshioka Osaka University, Yoshiki Higo Osaka University, Shinsuke Matsumoto Osaka University, Shinji Kusumoto Osaka University, Shinji Itoh Hitachi, Ltd., Research &Development Group, Phan Thi Thanh Huyen Hitachi, Ltd., Research &Development Group | ||
12:10 10mResearch paper | On Deprecated API Usages: an Exploratory Study of Top-starred Projects on GitHub Research Papers Pietro Cassieri University of Salerno, Simone Romano University of Salerno, Giuseppe Scanniello University of Salerno |