ASE 2023
Mon 11 - Fri 15 September 2023 Kirchberg, Luxembourg
Thu 14 Sep 2023 13:42 - 13:54 at Room C - Testing AI Systems 4

When deploying Deep Neural Networks (DNNs), developers often convert models from one deep learning framework to another (e.g., TensorFlow to PyTorch). However, this process is error-prone and can impact target model accuracy. To identify the extent of such impact, we perform and briefly present a differential analysis against three DNNs widely used for image recognition (MobileNetV2, ResNet101, and InceptionV3) converted across four well-known deep learning frameworks (PyTorch, Keras, TensorFlow (TF), and TFLite), which revealed numerous model crashes and output label discrepancies of up to 72%. To mitigate such errors, we present a novel approach towards fault localization and repair of buggy deep learning framework conversions, focusing on pre-trained image recognition models. Our technique consists of four stages of analysis: 1) conversion tools, 2) model parameters, 3) model hyperparameters, and 4) graph representation. In addition, we propose various strategies towards fault repair of the faults detected. We implement our technique on top of the Apache TVM deep learning compiler, and we test it by conducting a preliminary fault localization analysis for the conversion of InceptionV3 from TF to TFLite. Our approach detected a fault in a common DNN converter tool, which introduced precision errors in weights, reducing model accuracy. After our fault localization, we repaired the issue, reducing our conversion error to zero.

Thu 14 Sep

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

13:30 - 15:00
Testing AI Systems 4Research Papers / NIER Track at Room C
13:30
12m
Talk
Mutation-based Fault Localization of Deep Neural NetworksACM Distinguished Paper
Research Papers
Ali Ghanbari Iowa State University, Deepak-George Thomas Dept. of Computer Science, Iowa State University, Muhammad Arbab Arshad Dept. of Computer Science, Iowa State University, Hridesh Rajan Iowa State University
Pre-print
13:42
12m
Talk
Fault Localization for Buggy Deep Learning Framework Conversions in Image Recognition
NIER Track
Nikolaos Louloudakis University of Edinburgh, Perry Gibson University of Glasgow, José Cano University of Glasgow, Ajitha Rajan University of Edinburgh
Pre-print File Attached
13:54
12m
Talk
Towards Safe Automated Refactoring of Imperative Deep Learning Programs to Graph Execution
NIER Track
Raffi Khatchadourian City University of New York (CUNY) Hunter College, Tatiana Castro Vélez City University of New York (CUNY) Graduate Center, Mehdi Bagherzadeh Oakland University, Nan Jia City University of New York (CUNY) Graduate Center, Anita Raja City University of New York (CUNY) Hunter College
Pre-print Media Attached
14:06
12m
Talk
AutoConf : Automated Configuration of Unsupervised Learning Systems using Metamorphic Testing and Bayesian Optimization
Research Papers
Lwin Khin Shar Singapore Management University, Arda Goknil SINTEF Digital, Erik Johannes Husom SINTEF Digital, Sagar Sen , Yan Naing Tun Singapore Management University, Kisub Kim Singapore Management University, Singapore
File Attached
14:18
12m
Talk
An Intentional Forgetting-Driven Self-Healing Method For Deep Reinforcement Learning SystemsRecorded talk
Research Papers
Ahmed Haj Yahmed École Polytechnique de Montréal, Rached Bouchoucha Polytechnique Montréal, Houssem Ben Braiek Polytechnique Montréal, Foutse Khomh Polytechnique Montréal
Pre-print Media Attached
14:30
12m
Talk
A Majority Invariant Approach to Patch Robustness Certification for Deep Learning ModelsRecorded talk
NIER Track
Qilin Zhou City University of Hong Kong, Zhengyuan Wei City University of Hong Kong, Hong Kong, Haipeng Wang City University of Hong Kong, Wing-Kwong Chan City University of Hong Kong, Hong Kong
Pre-print Media Attached