Write a Blog >>
MET 2021
Wed 2 Jun 2021
co-located with ICSE 2021

The test oracle problem is one of the most challenging problems in software engineering. A growing body of research has examined the concept of Metamorphic Testing (MT), and has proven that MT can effectively alleviate the oracle problem and detect real bugs. Compared with most other testing methods, where the correctness of each individual test output is checked, MT has a different perspective on testing: it focuses on the relationships among the inputs and outputs of multiple executions of the software under test. MET: The International Workshop on Metamorphic Testing will bring together researchers and practitioners in academia and industry to discuss research results, experiences, and insights into MT. The ultimate goal of MET is to provide a platform for the discussion of novel ideas, new perspectives, new applications, and the state of research, related to or inspired by MT.

More information about MET can be found on: http://metwiki.net/MET21/

You're viewing the program in a time zone which is different from your device's time zone - change time zone

Conference Day
Wed 2 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:00 - 10:20
Opening SessionMET 2021 at MET Room
Chair(s): Pak-Lok PoonSchool of Engineering & Technology, Central Queensland University, Australia
10:00
20m
Day opening
Opening Remarks
MET 2021
Pak-Lok PoonSchool of Engineering & Technology, Central Queensland University, Australia
10:20 - 11:00
Session 1: Autonomous Driving and SimulationMET 2021 at MET Room
Chair(s): Pak-Lok PoonSchool of Engineering & Technology, Central Queensland University, Australia
10:20
20m
Short-paper
BMT: Behavior Driven Development-based MetamorphicTesting for Autonomous Driving Models
MET 2021
Yao DengMacquarie University, Guannan LouMacquarie University, Xi ZhengMacquarie University, Tianyi ZhangHarvard University, USA, Miryung KimUniversity of California at Los Angeles, USA, Huai LiuSwinburne University of Technology, Chen WangCSIRO DATA61, Tsong Yueh ChenSwinburne University of Technology
10:40
20m
Short-paper
Enhancing Euro NCAP Standards with Metamorphic Testing for Verification of Advanced Driver-Assistance Systems
MET 2021
Muhammad IqbalUniversity of Wollongong, Jia Cheng HanUniversity of Wollongong, Zhi Quan (George) ZhouUniversity of Wollongong, Australia, Dave ToweyUniversity of Nottingham Ningbo China
12:00 - 13:20
Session 2: Safety and Security and Metamorphic RelationsMET 2021 at MET Room
Chair(s): Xiaoyuan XieSchool of Computer Science, Wuhan University, China
12:00
30m
Long-paper
Robustness Evaluation of Stacked Generative Adversarial Networks using Metamorphic Testing
MET 2021
Hyejin ParkSchool of Information Technology, Monash University Malaysia, Taaha WaseemSchool of Information Technology, Monash University Malaysia, Wen Qi TeoSchool of Information Technology, Monash University Malaysia, Ying Hwei LowSchool of Information Technology, Monash University Malaysia, Mei Kuan LimSchool of Information Technology, Monash University Malaysia, Chun Yong ChongMonash University
12:30
30m
Long-paper
MT4NS: Metamorphic Testing for Network Scanning
MET 2021
Zhirui ZhangUniversity of Nottingham Ningbo China, Dave ToweyUniversity of Nottingham Ningbo China, Zhihao YingUniversity of Nottingham Ningbo China, Yifan ZhangUniversity of Nottingham Ningbo China, Zhi Quan (George) ZhouUniversity of Wollongong, Australia
13:00
20m
Short-paper
Follow-up Test Cases are Better Than Source Test Cases in Metamorphic Testing: A Preliminary Study
MET 2021
Zenghui ZhouBeihang University, Zheng ZhengBeihang University, Tsong Yueh ChenSwinburne University of Technology, Jinyi ZhouBeihang University, Kun QiuBeihang University
15:00 - 16:00
Keynote speechMET 2021 at MET Room
Chair(s): Sergio SeguraUniversidad de Sevilla, Pak-Lok PoonSchool of Engineering & Technology, Central Queensland University, Australia, Xiaoyuan XieSchool of Computer Science, Wuhan University, China
15:00
60m
Keynote
How do we go — to MET 2.0?
MET 2021
Adam PorterUniversity of Maryland
16:10 - 17:30
Session 3: Safety and Security and SimulationMET 2021 at MET Room
Chair(s): Xiaoyuan XieSchool of Computer Science, Wuhan University, China
16:10
30m
Long-paper
Metamorphic Detection of Repackaged Malware
MET 2021
Shirish SinghColumbia University, Gail KaiserColumbia University
16:40
20m
Short-paper
Metamorphic Testing on the Continuum of Verification and Validation of Simulation Models
MET 2021
M S RaunakNational Institute of Standards and Technology, Megan OlsenLoyola University Maryland
17:00
30m
Long-paper
Metamorphic Testing for Image-based Calcium Imaging Analysis Pipelines
MET 2021
Madeline DiepFraunhofer USA CESE, Rohan Reddy MekalaFraunhofer USA CESE, Ujjwal AyyangarFraunhofer USA CESE, Gudjon MagnussonFraunhofer USA CESE, Adam PorterFraunhofer USA CESE, Wolfgang LosertUniversity of Maryland, Mikael LindvallFraunhofer USA CESE
18:00 - 19:20
Session 4: Safety and Security and Simulation and Metamorphic RelationsMET 2021 at MET Room
Chair(s): Sergio SeguraUniversidad de Sevilla
18:00
30m
Long-paper
Applying Metamorphic Testing to Homomorphic Cryptography
MET 2021
Melvin WolfJacobs University Bremen, Jürgen SchönwälderJacobs University Bremen
18:30
20m
Short-paper
Towards Automated Metamorphic Test identification for Ocean System Models
MET 2021
Dilip Jagadeeshwarswamy HiremathGEOMAR, Helmholtz Centre for Ocean Research and Kiel University, Martin ClausGEOMAR, Helmholtz Centre for Ocean Research and Kiel University, Wilhelm HasselbringKiel University, Willi RathGEOMAR, Helmholtz Centre for Ocean Research
18:50
30m
Long-paper
Dreaming up Metamorphic Relations: Experiences from Three Fuzzer ToolsBest MET Paper Award
MET 2021
Andrei LascuImperial College London, Matt WindsorImperial College London, Alastair F. DonaldsonImperial College London, Tobias GrosserUniversity of Edinburgh, John WickersonImperial College London
19:20 - 19:40
Closing SessionMET 2021 at MET Room
Chair(s): Sergio SeguraUniversidad de Sevilla
19:20
20m
Day closing
Closing Remarks
MET 2021
Sergio SeguraUniversidad de Sevilla

Call for Papers

Metamorphic testing (MT) is a testing technique that exploits the relationships among the inputs and outputs of multiple executions of the program under test, so-called Metamorphic Relations (MRs). MT has been proven highly effective in testing programs that face the oracle problem, for which the correctness of individual output is difficult to determine. Since the introduction of MT two decades ago, the interest in this testing methodology has grown immensely with numerous applications in various domains such as machine learning, bioinformatics, computer graphics, simulation, search engines, decision support, cloud computing, databases, and compilers. The 6th International Workshop on Metamorphic Testing (MET) will bring together researchers and practitioners in academia and industry to discuss research results and experiences in MT. The ultimate goal of MET is to provide a platform for the discussion of novel ideas, new perspectives, new applications, and state of research, related to or inspired by MT.

TOPICS OF INTEREST

We invite original submissions on, but not limited to, the following topics on MT:

  • Emerging and innovative applications of MT (e.g., artificial intelligence).
  • Guidelines and techniques for the construction of MRs or MT test cases.
  • Prioritization and minimization of MRs or MT test cases.
  • Quality assessment mechanisms for MRs or MT test cases (e.g., metrics).
  • Automated generation of potential MRs.
  • Combination of MRs.
  • Formal methods involving MRs.
  • Case studies from practice.
  • Tools.
  • Surveys.
  • Empirical studies.
  • Integration/comparison with other techniques.
  • Novel applications, perspectives, or theories inspired by MT.

Accepted Papers

Title
Applying Metamorphic Testing to Homomorphic Cryptography
MET 2021
BMT: Behavior Driven Development-based MetamorphicTesting for Autonomous Driving Models
MET 2021
Closing Remarks
MET 2021
Dreaming up Metamorphic Relations: Experiences from Three Fuzzer ToolsBest MET Paper Award
MET 2021
Enhancing Euro NCAP Standards with Metamorphic Testing for Verification of Advanced Driver-Assistance Systems
MET 2021
Follow-up Test Cases are Better Than Source Test Cases in Metamorphic Testing: A Preliminary Study
MET 2021
MT4NS: Metamorphic Testing for Network Scanning
MET 2021
Metamorphic Detection of Repackaged Malware
MET 2021
Metamorphic Testing for Image-based Calcium Imaging Analysis Pipelines
MET 2021
Metamorphic Testing on the Continuum of Verification and Validation of Simulation Models
MET 2021
Opening Remarks
MET 2021
Robustness Evaluation of Stacked Generative Adversarial Networks using Metamorphic Testing
MET 2021
Towards Automated Metamorphic Test identification for Ocean System Models
MET 2021