DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs
Tue 10 May 2022 13:20 - 13:25 at ICSE room 3-odd hours - Program Analysis 4 Chair(s): Miguel Goulao
Thu 26 May 2022 11:15 - 11:20 at Room 306+307 - Papers 14: Program Analysis Chair(s): Frank Tip
Deep Neural Networks (DNNs) are used in a wide variety of applications. However, as in any software application, DNN-based apps are afflicted with bugs. Previous work observed that DNN bug fix patterns are different from traditional bug fix patterns. Furthermore, those buggy models are non-trivial to diagnose and fix due to inexplicit errors with several options to fix them. To support developers in locating and fixing bugs, we propose DeepDiagnosis, a novel debugging approach that localizes the faults, reports error symptoms and suggests fixes for DNN programs. In the first phase, our technique monitors a training model, periodically checking for eight types of error conditions. Then, in case of problems, it reports messages containing sufficient information to perform actionable repairs to the model. In the evaluation, we thoroughly examine 444 models – 53 real-world from GitHub and Stack Overflow, and 391 curated by AUTOTRAINER. DeepDiagnosis provides superior accuracy when compared to UMLUAT and DeepLocalize. Our technique is faster than AUTOTRAINER for fault localization. The results show that our approach can support additional types of models, while state-of-the-art was only able to handle classification ones. Our technique was able to report bugs that do not manifest as numerical errors during training. Also, it can provide actionable insights for fix whereas DeepLocalize can only report faults that lead to numerical errors during training. DeepDiagnosis manifests the best capabilities of fault detection, bug localization, and symptoms identification when compared to other approaches.
Mon 9 MayDisplayed time zone: Eastern Time (US & Canada) change
21:00 - 22:00 | Program Analysis 3Technical Track / SEIP - Software Engineering in Practice / Journal-First Papers at ICSE room 5-odd hours Chair(s): Travis Breaux Carnegie Mellon University | ||
21:00 5mTalk | Learning to Find Usages of Library Functions in Optimized Binaries Journal-First Papers Toufique Ahmed University of California at Davis, Prem Devanbu Department of Computer Science, University of California, Davis, Anand Ashok Sawant University of California, Davis Link to publication DOI Pre-print Media Attached | ||
21:05 5mTalk | InspectJS: Leveraging Code Similarity and User-Feedback for Effective Taint Specification Inference for JavaScript SEIP - Software Engineering in Practice Saikat Dutta University of Illinois at Urbana-Champaign, Diego Garbervetsky University of Buenos Aires and CONICET, Argentina, Shuvendu K. Lahiri Microsoft Research, Max Schaefer GitHub, Inc. DOI Pre-print Media Attached | ||
21:10 5mTalk | Static Inference Meets Deep Learning: A Hybrid Type Inference Approach for PythonNominated for Distinguished Paper Technical Track Yun Peng The Chinese University of Hong Kong, Cuiyun Gao Harbin Institute of Technology, Zongjie Li The Hong Kong University of Science and Technology, Bowei Gao Harbin Institute of Technology, Shenzhen, David Lo Singapore Management University, Qirun Zhang Georgia Institute of Technology, USA, Michael Lyu The Chinese University of Hong Kong DOI Pre-print Media Attached | ||
21:15 5mTalk | DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs Technical Track Mohammad Wardat Dept. of Computer Science, Iowa State University, Breno Dantas Cruz Dept. of Computer Science, Iowa State University, Wei Le Iowa State University, Hridesh Rajan Iowa State University Pre-print Media Attached | ||
21:20 5mTalk | Striking a Balance: Pruning False-Positives from Static Call GraphsNominated for Distinguished Paper Technical Track Akshay Utture University of California, Los Angeles (UCLA), Shuyang Liu University of California, Los Angeles, Christian Gram Kalhauge Technical University of Denmark, Jens Palsberg University of California at Los Angeles DOI Pre-print Media Attached | ||
Tue 10 MayDisplayed time zone: Eastern Time (US & Canada) change
Thu 26 MayDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | Papers 14: Program AnalysisTechnical Track / SEIP - Software Engineering in Practice / Journal-First Papers at Room 306+307 Chair(s): Frank Tip Northeastern University | ||
11:00 5mTalk | Static Inference Meets Deep Learning: A Hybrid Type Inference Approach for PythonNominated for Distinguished Paper Technical Track Yun Peng The Chinese University of Hong Kong, Cuiyun Gao Harbin Institute of Technology, Zongjie Li The Hong Kong University of Science and Technology, Bowei Gao Harbin Institute of Technology, Shenzhen, David Lo Singapore Management University, Qirun Zhang Georgia Institute of Technology, USA, Michael Lyu The Chinese University of Hong Kong DOI Pre-print Media Attached | ||
11:05 5mTalk | TaintBench: Automatic Real-World Malware Benchmarking of Android Taint Analyses Journal-First Papers Linghui Luo Amazon Web Services, Felix Pauck Paderborn University, Germany, Goran Piskachev Fraunhofer IEM, Manuel Benz Paderborn University, Ivan Pashchenko University of Trento, Martin Mory Paderborn University, Eric Bodden , Ben Hermann Technical University Dortmund, Fabio Massacci University of Trento; Vrije Universiteit Amsterdam Link to publication DOI Pre-print Media Attached File Attached | ||
11:10 5mTalk | InspectJS: Leveraging Code Similarity and User-Feedback for Effective Taint Specification Inference for JavaScript SEIP - Software Engineering in Practice Saikat Dutta University of Illinois at Urbana-Champaign, Diego Garbervetsky University of Buenos Aires and CONICET, Argentina, Shuvendu K. Lahiri Microsoft Research, Max Schaefer GitHub, Inc. DOI Pre-print Media Attached | ||
11:15 5mTalk | DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs Technical Track Mohammad Wardat Dept. of Computer Science, Iowa State University, Breno Dantas Cruz Dept. of Computer Science, Iowa State University, Wei Le Iowa State University, Hridesh Rajan Iowa State University Pre-print Media Attached | ||
11:20 5mTalk | Inference and Test Generation Using Program Invariants in Chemical Reaction Networks Technical Track Michael C. Gerten Iowa State University, Alexis L. Marsh Iowa State University, James I. Lathrop Iowa State University, Myra Cohen Iowa State University, Andrew S. Miner Iowa State University, Titus H. Klinge Drake University DOI Pre-print Media Attached | ||
11:25 5mTalk | PUS: A Fast and Highly Efficient Solver for Inclusion-based Pointer AnalysisDistinguished Paper Award Technical Track Peiming Liu Texas A&M University, Yanze Li University of British Columbia, Bradley Swain Texas A&M University, Jeff Huang Texas A&M University Pre-print Media Attached | ||
11:30 5mTalk | Fast and Precise Application Code Analysis using a Partial Library Technical Track Akshay Utture University of California, Los Angeles (UCLA), Jens Palsberg University of California at Los Angeles DOI Pre-print Media Attached | ||