ICSE 2024
Fri 12 - Sun 21 April 2024 Lisbon, Portugal
Fri 19 Apr 2024 16:00 - 16:15 at Sophia de Mello Breyner Andresen - Testing of AI systems Chair(s): Aldeida Aleti

Deep neural networks (DNN) are being used in a wide range of applications including safety-critical systems. Several DNN test generation approaches have been proposed to generate fault-revealing test inputs. However, the existing test generation approaches do not systematically cover the input data distribution to test DNNs with diverse inputs, and none of the approaches investigate the relationship between rare inputs and faults. We propose CIT4DNN, an automated black-box approach to generate DNN test sets that are feature-diverse and fault-revealing. CIT4DNN constructs diverse test sets by applying combinatorial interaction testing to the latent space of generative models and formulates constraints over the geometry of the latent space to generate rare and fault-revealing test inputs. Evaluation on a range of datasets and models shows that CIT4DNN generated tests are more feature diverse than the state-of-the-art, and can target rare fault-revealing testing inputs more effectively than existing methods.

Fri 19 Apr

Displayed time zone: Lisbon change

16:00 - 17:30
Testing of AI systemsResearch Track / Journal-first Papers at Sophia de Mello Breyner Andresen
Chair(s): Aldeida Aleti Monash University
16:00
15m
Talk
CIT4DNN: Generating Diverse and Rare Inputs for Neural Networks Using Latent Space Combinatorial Testing
Research Track
Swaroopa Dola University of Virginia, Rory McDaniel University of Virginia, Matthew B Dwyer University of Virginia, Mary Lou Soffa University of Virginia
16:15
15m
Talk
Knowledge Graph Driven Inference Testing for Question Answering Software
Research Track
Jun Wang Nanjing University, Yanhui Li Nanjing University, Zhifei Chen Nanjing University, Lin Chen Nanjing University, Xiaofang Zhang Soochow University, Yuming Zhou Nanjing University
16:30
15m
Talk
DeepSample: DNN sampling-based testing for operational accuracy assessment
Research Track
Antonio Guerriero Università di Napoli Federico II, Roberto Pietrantuono Università di Napoli Federico II, Stefano Russo Università di Napoli Federico II
Pre-print
16:45
15m
Talk
MAFT: Efficient Model-Agnostic Fairness Testing for Deep Neural Networks via Zero-Order Gradient Search
Research Track
Zhaohui Wang East China Normal University, Min Zhang East China Normal University, Jingran Yang East China Normal University, ShaoBojie East China Normal University, Min Zhang East China Normal University
17:00
7m
Talk
DeepManeuver: Adversarial Test Generation for Trajectory Manipulation of Autonomous Vehicles
Journal-first Papers
Meriel von Stein University of Virginia, Sebastian Elbaum University of Virginia, David Shriver Software Engineering Institute
17:07
7m
Talk
Finding Deviated Behaviors of the Compressed DNN Models for Image Classifications
Journal-first Papers
Yongqiang Tian The Hong Kong University of Science and Technology; University of Waterloo, Wuqi Zhang The Hong Kong University of Science and Technology, Ming Wen Huazhong University of Science and Technology, Shing-Chi Cheung Hong Kong University of Science and Technology, Chengnian Sun University of Waterloo, Shiqing Ma UMass Amherst, Yu Jiang Tsinghua University
Link to publication DOI
17:14
7m
Talk
Identifying the Hazard Boundary of ML-enabled Autonomous Systems Using Cooperative Co-Evolutionary Search
Journal-first Papers
Sepehr Sharifi University of Ottawa, Donghwan Shin University of Sheffield, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland, Nathan Aschbacher Auxon Corporation
Hide past events