We present SAFE, a tool based on a black-box approach to automatically characterize the root causes of Deep Neural Network (DNN) failures. SAFE relies on VGGNet-16, a transfer learning model pre-trained on ImageNet, to extract the features from error-inducing images. After feature extraction, SAFE applies a density-based clustering algorithm to discover arbitrarily shaped clusters of images modeling plausible causes of failures. By relying on the identified clusters, SAFE can select a set of additional images to be used to retrain and improve the DNN efficiently. Empirical results show the potential of SAFE in identifying different root causes of DNN failures based on case studies in the automotive domain. It also yields significant improvements in DNN accuracy after retraining while saving considerable execution time and memory compared to alternatives. A demo video of SAFE is available at: https://youtu.be/RkTxNi3DVMM
Fri 19 AprDisplayed time zone: Lisbon change
11:00 - 12:30 | Testing with and for AI 1Research Track / Journal-first Papers / Demonstrations at Sophia de Mello Breyner Andresen Chair(s): Peter Rigby Concordia University; Meta | ||
11:00 15mTalk | Prompting Is All Your Need: Automated Android Bug Replay with Large Language Models Research Track | ||
11:15 15mTalk | Towards Reliable AI: Adequacy Metrics for Ensuring the Quality of System-level Testing of Autonomous Vehicles Research Track | ||
11:30 15mTalk | Learning-based Widget Matching for Migrating GUI Test Cases Research Track Yakun Zhang Peking University, Wenjie Zhang Peking University, Dezhi Ran Peking University, Qihao Zhu Peking University, Chengfeng Dou Peking University, Dan Hao Peking University, Tao Xie Peking University, Lu Zhang Peking University | ||
11:45 7mTalk | A Search-Based Testing Approach for Deep Reinforcement Learning Agents Journal-first Papers Amirhossein Zolfagharian University of Ottawa - School of Electrical Engineering & Computer Science (EECS), Manel Abdellatif Software and Information Technology Engineering Department, École de Technologie Supérieure, Mojtaba Bagherzadeh Cisco, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland, Ramesh S | ||
11:52 7mTalk | StubCoder: Automated Generation and Repair of Stub Code for Mock Objects Journal-first Papers Hengcheng Zhu The Hong Kong University of Science and Technology, Lili Wei McGill University, Valerio Terragni University of Auckland, Yepang Liu Southern University of Science and Technology, Shing-Chi Cheung Hong Kong University of Science and Technology, Jiarong Wu , Qin Sheng WeBank Co Ltd, Bing Zhang WeBank Co. Ltd., Lihong Song WeBank Co. Ltd. Link to publication DOI Authorizer link Pre-print | ||
11:59 7mTalk | Testing of Deep Reinforcement Learning Agents with Surrogate Models Journal-first Papers | ||
12:06 7mTalk | Model vs System Level Testing of Autonomous Driving Systems: A Replication and Extension Study Journal-first Papers Andrea Stocco Technical University of Munich, fortiss, Brian Pulfer University of Geneva, Paolo Tonella USI Lugano | ||
12:13 7mTalk | SAFE: Safety Analysis and Retraining of DNNs Demonstrations Mohammed Attaoui University of Luxembourg, Fabrizio Pastore University of Luxembourg, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland Pre-print | ||
12:20 7mTalk | MutaBot: A Mutation Testing Approach for Chatbots Demonstrations Michael Ferdinando Urrico University of Milano - Bicocca, Diego Clerissi University of Milano-Bicocca, Leonardo Mariani University of Milano-Bicocca DOI Pre-print Media Attached | ||