Write a Blog >>
ASE 2021
Sun 14 - Sat 20 November 2021 Australia
Wed 17 Nov 2021 09:00 - 09:20 at Kangaroo - Learning I Chair(s): Denys Poshyvanyk

Deep Learning (DL) components are routinely integrated into software systems that need to perform complex tasks such as image or natural language processing. The adequacy of the test data used to test such systems can be assessed by their ability to expose artificially injected faults (mutations) that simulate real DL faults.

In this paper, we describe an approach to automatically generate new test inputs that can be used to augment the existing test set so that its capability to detect DL mutations increases. Our tool DeepMetis implements a search based input generation strategy. To account for the non-determinism of the training and the mutation processes, our fitness function involves multiple instances of the DL model under test. Experimental results show that DeepMetis is effective at augmenting the given test set, increasing its capability to detect mutants by 63% on average. A leave-one-out experiment shows that the augmented test set is capable to expose unseen mutants, which simulate the occurrence of yet undetected faults.

Wed 17 Nov

Displayed time zone: Hobart change

09:00 - 10:00
Learning INIER track / Research Papers / Tool Demonstrations at Kangaroo
Chair(s): Denys Poshyvanyk William and Mary
DeepMetis: Augmenting a Deep Learning Test Set to Increase its Mutation Score
Research Papers
Vincenzo Riccio USI Lugano, Nargiz Humbatova Università della Svizzera Italiana (USI), Gunel Jahangirova USI Lugano, Paolo Tonella USI Lugano
Efficient state synchronisation in model-based testing through reinforcement learning
Research Papers
Uraz Cengiz Türker University of Leicester, UK, Robert Hierons University of Sheffield, Mohammad Reza Mousavi King's College London, Ivan Tyukin University of Leicester
What do pre-trained code models know about code?
NIER track
Anjan Karmakar Free University of Bozen-Bolzano, Romain Robbes
DEVIATE: A Deep Learning Variance Testing Framework
Tool Demonstrations
Viet Hung Pham University of Waterloo, Mijung Kim Purdue University, Lin Tan Purdue University, Yaoliang Yu University of Waterloo, Nachiappan Nagappan Microsoft Research