FSE 2025
Mon 23 - Fri 27 June 2025 Trondheim, Norway
Mon 23 Jun 2025 17:20 - 17:40 at Cosmos 3A - Testing 2 Chair(s): Miryung Kim

Metamorphic Testing (MT) alleviates the oracle problem by defining oracles based on metamorphic relations (MRs) that govern multiple related inputs and their outputs. However, designing MRs is challenging, as it requires domain-specific knowledge. This hinders the widespread adoption of MT. We observe that developer-written test cases can embed domain knowledge that encodes MRs. Such encoded MRs could be synthesized for testing not only their original programs but also other programs that share similar functionalities.

In this article, we propose MR-Scout to automatically synthesize MRs from test cases in open-source software (OSS) projects. MR-Scout first discovers MR-encoded test cases (MTCs), and then synthesizes the encoded MRs into parameterized methods (called codified MRs), and filters out MRs that demonstrate poor quality for new test case generation. MR-Scout discovered over 11,000 MTCs from 701 OSS projects. Experimental results show that over 97% of codified MRs are of high quality for automated test case generation, demonstrating the practical applicability of MR-Scout. Furthermore, codified-MRs-based tests effectively enhance the test adequacy of programs with developer-written tests, leading to 13.52% and 9.42% increases in line coverage and mutation score, respectively. Our qualitative study shows that 55.76% to 76.92% of codified MRs are easily comprehensible for developers.

Mon 23 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

16:00 - 18:00
Testing 2Journal First / Research Papers at Cosmos 3A
Chair(s): Miryung Kim UCLA and Amazon Web Services
16:00
20m
Talk
Search-based DNN Testing and Retraining with GAN-enhanced Simulations
Journal First
Mohammed Attaoui University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland
16:20
20m
Talk
TEASMA: A Practical Methodology for Test Adequacy Assessment of Deep Neural Networks
Journal First
Amin Abbasishahkoo The School of EECS, University of Ottawa, Mahboubeh Dadkhah University of Ottawa, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland, Dayi Lin Centre for Software Excellence, Huawei Canada
16:40
20m
Talk
VLATest: Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation
Research Papers
Zhijie Wang University of Alberta, Zhehua Zhou University of Macau, Norman Song , Yuheng Huang The University of Tokyo, Zhan Shu University of Alberta, Lei Ma The University of Tokyo & University of Alberta
DOI Pre-print
17:00
20m
Talk
DRWASI: LLM-assisted Differential Testing for WebAssembly System Interface Implementations
Journal First
Yixuan Zhang Peking University, Ningyu He Hong Kong Polytechnic University, Jianting Gao Huazhong University of Science and Technology, Shangtong Cao Beijing University of Posts and Telecommunications, Kaibo Liu Peking University, Haoyu Wang Huazhong University of Science and Technology, Yun Ma Peking University, Gang Huang Peking University, Xuanzhe Liu Peking University
17:20
20m
Talk
MR-Scout: Automated Synthesis of Metamorphic Relations from Existing Test Cases
Journal First
Congying Xu The Hong Kong University of Science and Technology, China, Valerio Terragni University of Auckland, Hengcheng Zhu The Hong Kong University of Science and Technology, Jiarong Wu , Shing-Chi Cheung Hong Kong University of Science and Technology
17:40
20m
Talk
UnitCon: Synthesizing Targeted Unit Tests for Java Runtime Exceptions
Research Papers
Sujin Jang KAIST, Yeonhee Ryou KAIST, Heewon Lee KAIST, Korea, South (The Republic of), Kihong Heo KAIST
DOI

Information for Participants
Mon 23 Jun 2025 16:00 - 18:00 at Cosmos 3A - Testing 2 Chair(s): Miryung Kim
Info for room Cosmos 3A:

Cosmos 3A is the first room in the Cosmos 3 wing.

When facing the main Cosmos Hall, access to the Cosmos 3 wing is on the left, close to the stairs. The area is accessed through a large door with the number “3”, which will stay open during the event.