Write a Blog >>
MET 2021
Wed 2 Jun 2021
co-located with ICSE 2021

The test oracle problem is one of the most challenging problems in software engineering. A growing body of research has examined the concept of Metamorphic Testing (MT), and has proven that MT can effectively alleviate the oracle problem and detect real bugs. Compared with most other testing methods, where the correctness of each individual test output is checked, MT has a different perspective on testing: it focuses on the relationships among the inputs and outputs of multiple executions of the software under test. MET: The International Workshop on Metamorphic Testing will bring together researchers and practitioners in academia and industry to discuss research results, experiences, and insights into MT. The ultimate goal of MET is to provide a platform for the discussion of novel ideas, new perspectives, new applications, and the state of research, related to or inspired by MT.

More information about MET can be found on: http://metwiki.net/MET21/

You're viewing the program in a time zone which is different from your device's time zone - change time zone

Wed 2 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:00 - 10:20
Opening SessionMET 2021 at MET Room
Chair(s): Pak-Lok Poon School of Engineering & Technology, Central Queensland University, Australia
10:00
20m
Day opening
Opening Remarks
MET 2021
Pak-Lok Poon School of Engineering & Technology, Central Queensland University, Australia
Media Attached
10:20 - 11:00
Session 1: Autonomous Driving and SimulationMET 2021 at MET Room
Chair(s): Pak-Lok Poon School of Engineering & Technology, Central Queensland University, Australia
10:20
20m
Short-paper
BMT: Behavior Driven Development-based MetamorphicTesting for Autonomous Driving Models
MET 2021
Yao Deng Macquarie University, Guannan Lou Macquarie University, Xi Zheng Macquarie University, Tianyi Zhang Harvard University, USA, Miryung Kim University of California at Los Angeles, USA, Huai Liu Swinburne University of Technology, Chen Wang CSIRO DATA61, Tsong Yueh Chen Swinburne University of Technology
Media Attached
10:40
20m
Short-paper
Enhancing Euro NCAP Standards with Metamorphic Testing for Verification of Advanced Driver-Assistance Systems
MET 2021
Muhammad Iqbal University of Wollongong, Jia Cheng Han University of Wollongong, Zhi Quan (George) Zhou University of Wollongong, Australia, Dave Towey University of Nottingham Ningbo China
Media Attached
12:00 - 13:20
Session 2: Safety and Security and Metamorphic RelationsMET 2021 at MET Room
Chair(s): Xiaoyuan Xie School of Computer Science, Wuhan University, China
12:00
30m
Long-paper
Robustness Evaluation of Stacked Generative Adversarial Networks using Metamorphic Testing
MET 2021
Hyejin Park School of Information Technology, Monash University Malaysia, Taaha Waseem School of Information Technology, Monash University Malaysia, Wen Qi Teo School of Information Technology, Monash University Malaysia, Ying Hwei Low School of Information Technology, Monash University Malaysia, Mei Kuan Lim School of Information Technology, Monash University Malaysia, Chun Yong Chong Monash University
Media Attached
12:30
30m
Long-paper
MT4NS: Metamorphic Testing for Network Scanning
MET 2021
Zhirui Zhang University of Nottingham Ningbo China, Dave Towey University of Nottingham Ningbo China, Zhihao Ying University of Nottingham Ningbo China, Yifan Zhang University of Nottingham Ningbo China, Zhi Quan (George) Zhou University of Wollongong, Australia
Media Attached
13:00
20m
Short-paper
Follow-up Test Cases are Better Than Source Test Cases in Metamorphic Testing: A Preliminary Study
MET 2021
Zenghui Zhou Beihang University, Zheng Zheng Beihang University, Tsong Yueh Chen Swinburne University of Technology, Jinyi Zhou Beihang University, Kun Qiu Beihang University
Media Attached
15:00 - 16:00
Keynote speechMET 2021 at MET Room
Chair(s): Pak-Lok Poon School of Engineering & Technology, Central Queensland University, Australia, Sergio Segura Universidad de Sevilla, Xiaoyuan Xie School of Computer Science, Wuhan University, China
15:00
60m
Keynote
How do we go — to MET 2.0?
MET 2021
Adam Porter University of Maryland
Media Attached
16:10 - 17:30
Session 3: Safety and Security and SimulationMET 2021 at MET Room
Chair(s): Xiaoyuan Xie School of Computer Science, Wuhan University, China
16:10
30m
Long-paper
Metamorphic Detection of Repackaged Malware
MET 2021
Shirish Singh Columbia University, Gail Kaiser Columbia University
Media Attached
16:40
20m
Short-paper
Metamorphic Testing on the Continuum of Verification and Validation of Simulation Models
MET 2021
M S Raunak National Institute of Standards and Technology, Megan Olsen Loyola University Maryland
Media Attached
17:00
30m
Long-paper
Metamorphic Testing for Image-based Calcium Imaging Analysis Pipelines
MET 2021
Madeline Diep Fraunhofer USA CESE, Rohan Reddy Mekala Fraunhofer USA CESE, Ujjwal Ayyangar Fraunhofer USA CESE, Gudjon Magnusson Fraunhofer USA CESE, Adam Porter Fraunhofer USA CESE, Wolfgang Losert University of Maryland, Mikael Lindvall Fraunhofer USA CESE
Media Attached
18:00 - 19:20
Session 4: Safety and Security and Simulation and Metamorphic RelationsMET 2021 at MET Room
Chair(s): Sergio Segura Universidad de Sevilla
18:00
30m
Long-paper
Applying Metamorphic Testing to Homomorphic Cryptography
MET 2021
Melvin Wolf Jacobs University Bremen, Jürgen Schönwälder Jacobs University Bremen
Media Attached
18:30
20m
Short-paper
Towards Automated Metamorphic Test identification for Ocean System Models
MET 2021
Dilip Jagadeeshwarswamy Hiremath GEOMAR, Helmholtz Centre for Ocean Research and Kiel University, Martin Claus GEOMAR, Helmholtz Centre for Ocean Research and Kiel University, Wilhelm Hasselbring Kiel University, Willi Rath GEOMAR, Helmholtz Centre for Ocean Research
Media Attached
18:50
30m
Long-paper
Dreaming up Metamorphic Relations: Experiences from Three Fuzzer ToolsBest MET Paper Award
MET 2021
Andrei Lascu Imperial College London, Matt Windsor Imperial College London, Alastair F. Donaldson Imperial College London, Tobias Grosser University of Edinburgh, John Wickerson Imperial College London
Media Attached
19:20 - 19:40
Closing SessionMET 2021 at MET Room
Chair(s): Sergio Segura Universidad de Sevilla
19:20
20m
Day closing
Closing Remarks
MET 2021
Sergio Segura Universidad de Sevilla
Media Attached

Call for Papers

Metamorphic testing (MT) is a testing technique that exploits the relationships among the inputs and outputs of multiple executions of the program under test, so-called Metamorphic Relations (MRs). MT has been proven highly effective in testing programs that face the oracle problem, for which the correctness of individual output is difficult to determine. Since the introduction of MT two decades ago, the interest in this testing methodology has grown immensely with numerous applications in various domains such as machine learning, bioinformatics, computer graphics, simulation, search engines, decision support, cloud computing, databases, and compilers. The 6th International Workshop on Metamorphic Testing (MET) will bring together researchers and practitioners in academia and industry to discuss research results and experiences in MT. The ultimate goal of MET is to provide a platform for the discussion of novel ideas, new perspectives, new applications, and state of research, related to or inspired by MT.

TOPICS OF INTEREST

We invite original submissions on, but not limited to, the following topics on MT:

  • Emerging and innovative applications of MT (e.g., artificial intelligence).
  • Guidelines and techniques for the construction of MRs or MT test cases.
  • Prioritization and minimization of MRs or MT test cases.
  • Quality assessment mechanisms for MRs or MT test cases (e.g., metrics).
  • Automated generation of potential MRs.
  • Combination of MRs.
  • Formal methods involving MRs.
  • Case studies from practice.
  • Tools.
  • Surveys.
  • Empirical studies.
  • Integration/comparison with other techniques.
  • Novel applications, perspectives, or theories inspired by MT.

Accepted Papers

Title
Applying Metamorphic Testing to Homomorphic Cryptography
MET 2021
Media Attached
BMT: Behavior Driven Development-based MetamorphicTesting for Autonomous Driving Models
MET 2021
Media Attached
Closing Remarks
MET 2021
Media Attached
Dreaming up Metamorphic Relations: Experiences from Three Fuzzer ToolsBest MET Paper Award
MET 2021
Media Attached
Enhancing Euro NCAP Standards with Metamorphic Testing for Verification of Advanced Driver-Assistance Systems
MET 2021
Media Attached
Follow-up Test Cases are Better Than Source Test Cases in Metamorphic Testing: A Preliminary Study
MET 2021
Media Attached
MT4NS: Metamorphic Testing for Network Scanning
MET 2021
Media Attached
Metamorphic Detection of Repackaged Malware
MET 2021
Media Attached
Metamorphic Testing for Image-based Calcium Imaging Analysis Pipelines
MET 2021
Media Attached
Metamorphic Testing on the Continuum of Verification and Validation of Simulation Models
MET 2021
Media Attached
Opening Remarks
MET 2021
Media Attached
Robustness Evaluation of Stacked Generative Adversarial Networks using Metamorphic Testing
MET 2021
Media Attached
Towards Automated Metamorphic Test identification for Ocean System Models
MET 2021
Media Attached