Write a Blog >>
ASE 2021
Sun 14 - Sat 20 November 2021 Australia
Thu 18 Nov 2021 22:40 - 22:50 at Kangaroo - Analysis III Chair(s): Jifeng Xuan

Metamorphic testing is a well-established testing technique that has been successfully applied in various domains, including testing deep learning models to assess their robustness against data noise or malicious input. Currently, metamorphic testing approaches for machine learning (ML) models focused on image processing and object recognition tasks. Hence, these approaches cannot be applied to ML targeting program analysis tasks. In this paper, we extend metamorphic testing approaches for ML models targeting software programs. We present Lampion, a novel testing framework that applies (semantics preserving) metamorphic transformations on the test datasets. Lampion produces new code snippets equivalent to the original test set but different in their identifiers or syntactic structure. We evaluate Lampion against CodeBERT, a state-of-the-art ML model for Code-To-Text tasks that creates Javadoc summaries for given Java methods. Our results show that simple transformations significantly impact the target model behavior, providing additional information on the models reasoning apart from the classic performance metric.

Thu 18 Nov

Displayed time zone: Hobart change

22:00 - 23:00
Analysis IIINIER track / Research Papers at Kangaroo
Chair(s): Jifeng Xuan Wuhan University
22:00
20m
Talk
Binary Diffing as a Network Alignment Problem via Belief Propagation
Research Papers
Elie Mengin SAMM, EA 4543 - Université Paris 1 Panthéon-Sorbonne, Fabrice Rossi CEREMADE, CNRS, UMR 7534 - Université Paris-Dauphine, PSL University
22:20
20m
Talk
CiFi: Versatile Analysis of Class and Field ImmutabilityACM Distinguished Paper
Research Papers
Tobias Roth Technische Universität Darmstadt, Dominik Helm Technische Universität Darmstadt, Michael Reif Technische Universität Darmstadt, Mira Mezini Technische Universität Darmstadt
22:40
10m
Talk
Assessing Robustness of ML-Based Program Analysis Tools using Metamorphic Program Transformations
NIER track
Leonhard Applis Delft University of Technology, Annibale Panichella Delft University of Technology, Arie van Deursen Delft University of Technology, Netherlands
Pre-print
22:50
10m
Talk
Defeating program analysis techniques via Ambiguous Translation
NIER track
Chijung Jung University of Virginia, Doowon Kim University of Tennessee, Knoxville, Weihang Wang University at Buffalo, SUNY, Yunhui Zheng IBM Research, Kyu Hyung Lee University of Georgia, Yonghwi Kwon University of Virginia