Write a Blog >>
ISSTA 2021
Sun 11 - Sat 17 July 2021 Online
co-located with ECOOP and ISSTA 2021

Deep Learning (DL) has been successfully applied to a wide range of application domains, including safety-critical ones. Several DL testing approaches have been recently proposed in the literature but none of them aims to assess how different interpretable features of the generated inputs affect the system’s behaviour.

In this paper, we resort to Illumination Search to find the highest-performing test cases (i.e., misbehaving and closest to misbehaving), spread across the cells of a map representing the feature space of the system. We introduce a methodology that guides the users of our approach in the tasks of identifying and quantifying the dimensions of the feature space for a given domain. We developed DeepHyperion, a search-based tool for DL systems that illuminates, i.e., explores at large, the feature space, by providing developers with an interpretable feature map where automatically generated inputs are placed along with information about the exposed behaviours.

DeepHyperion_Presentation (Tahereh_Zohdinasab_ISSTA21.pdf)11.85MiB

Wed 14 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

19:40 - 20:20
Session 4 (time band 1) Testing Deep Learning Systems 2Technical Papers at ISSTA 2
Chair(s): Sebastian Elbaum University of Virginia
19:40
20m
Talk
DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search
Technical Papers
Tahereh Zohdinasab USI Lugano, Vincenzo Riccio USI Lugano, Alessio Gambi University of Passau, Paolo Tonella USI Lugano
DOI File Attached
20:00
20m
Talk
Automatic Test Suite Generation for Key-Points Detection DNNs using Many-Objective Search (Experience Paper)
Technical Papers
Fitash Ul Haq University of Luxembourg, Donghwan Shin University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa, Thomas Stifter IEE, Jun Wang Post Luxembourg
DOI

Thu 15 Jul

Displayed time zone: Brussels, Copenhagen, Madrid, Paris change

09:10 - 10:50
Session 9 (time band 3) Testing Deep Learning Systems 3Technical Papers at ISSTA 1
Chair(s): Mauro Pezze USI Lugano; Schaffhausen Institute of Technology
09:10
20m
Talk
Attack as Defense: Characterizing Adversarial Examples using Robustness
Technical Papers
Zhe Zhao ShanghaiTech University, Guangke Chen ShanghaiTech University, Jingyi Wang Zhejiang University, Yiwei Yang ShanghaiTech University, Fu Song ShanghaiTech University, Jun Sun Singapore Management University
DOI Media Attached
09:30
20m
Talk
Exposing Previously Undetectable Faults in Deep Neural Networks
Technical Papers
Isaac Dunn University of Oxford, Hadrien Pouget University of Oxford, Daniel Kroening Amazon, Tom Melham University of Oxford
DOI Pre-print Media Attached
09:50
20m
Talk
Automatic Test Suite Generation for Key-Points Detection DNNs using Many-Objective Search (Experience Paper)
Technical Papers
Fitash Ul Haq University of Luxembourg, Donghwan Shin University of Luxembourg, Lionel Briand University of Luxembourg; University of Ottawa, Thomas Stifter IEE, Jun Wang Post Luxembourg
DOI
10:10
20m
Talk
DeepHyperion: Exploring the Feature Space of Deep Learning-Based Systems through Illumination Search
Technical Papers
Tahereh Zohdinasab USI Lugano, Vincenzo Riccio USI Lugano, Alessio Gambi University of Passau, Paolo Tonella USI Lugano
DOI File Attached
10:30
20m
Talk
DeepCrime: Mutation Testing of Deep Learning Systems Based on Real Faults
Technical Papers
Nargiz Humbatova USI Lugano, Gunel Jahangirova USI Lugano, Paolo Tonella USI Lugano
DOI