Efficient Understanding of Machine Learning Model Mispredictions
Mispredictions by machine learning components can have severe consequences, especially in safety-critical and mission-critical software systems. Therefore, understanding and debugging these mispredictions is a crucial part of the development process for systems that use machine learning components. Previous research has successfully applied methods that identify when a model’s predictions may be unreliable by generating a rule set that links feature values to prediction errors. However, current state-of-the-art rule set approaches require significant computational resources, particularly for large data sets.
To address these high computational demands, we propose a strategy to identify and focus only on the most influential features that lead to mispredictions. Additionally, to improve the accuracy of mispredictions diagnosis, we replace traditional rule-based approaches with decision tree learning. We evaluate our tool \MMDPP{} across 11 diverse real-world data sets. The results show that focusing on influential features with decision trees improves the accuracy of misprediction explanations, while significantly reducing computational demands in all scenarios. Thus, \MMDPP{} produces better results much faster, making it more efficient and effective for generating misprediction explanations.
Mon 17 NovDisplayed time zone: Seoul change
11:00 - 12:30 | SE4AI & AI4SE 1Research Papers / Journal-First at Vista Chair(s): Zhou Yang University of Alberta, Alberta Machine Intelligence Institute | ||
11:00 10mTalk | TensorGuard: Gradient-Based Model Fingerprinting for LLM Similarity Detection and Family Classification Research Papers Zehao Wu Huazhong University of Science and Technology, Yanjie Zhao Huazhong University of Science and Technology, Haoyu Wang Huazhong University of Science and Technology | ||
11:10 10mTalk | Root Cause Analysis of RISC-V Build Failures via LLM and MCTS Reasoning Research Papers Weipeng Shuai Institute of Software, Chinese Academy of Sciences, Jie Liu Institute of Software, Chinese Academy of Sciences, Zhirou Ma Institute of Software, Chinese Academy of Sciences, Liangyi Kang Institute of Software, Chinese Academy of Sciences, Zehua Wang Institute of Software, Chinese Academy of Sciences, Shuai Wang Institute of Software, Chinese Academy of Sciences, Dan Ye Institute of Software at Chinese Academy of Sciences, Hui Li , Wei Wang Institute of Software at Chinese Academy of Sciences, Jiaxin Zhu Institute of Software at Chinese Academy of Sciences | ||
11:20 10mTalk | An Empirical Study of Knowledge Transfer in AI Pair Programming Research Papers Alisa Carla Welter Saarland University, Niklas Schneider Saarland University, Tobias Dick Saarland University, Kallistos Weis Saarland University, Christof Tinnes Siemens AG, Marvin Wyrich Saarland University, Sven Apel Saarland University | ||
11:30 10mTalk | Efficient Understanding of Machine Learning Model Mispredictions Research Papers Martin Eberlein Humboldt-Universtität zu Berlin, Jürgen Cito TU Wien, Lars Grunske Humboldt-Universität zu Berlin | ||
11:40 10mTalk | Can Mamba Be Better? An Experimental Evaluation of Mamba in Code Intelligence Research Papers Shuo Liu City University of Hong Kong, Jacky Keung City University of Hong Kong, Zhen Yang Shandong University, Zhenyu Mao City University of Hong Kong, Yicheng Sun City University of Hong Kong | ||
11:50 10mTalk | "My productivity is boosted, but ..." Demystifying Users’ Perception on AI Coding Assistants Research Papers Yunbo Lyu Singapore Management University, Zhou Yang University of Alberta, Alberta Machine Intelligence Institute , Jieke Shi Singapore Management University, Chang Jianming , Yue Liu Monash University, David Lo Singapore Management University Pre-print | ||
12:00 10mTalk | HFUZZER: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing Research Papers Yukai Zhao , Menghan Wu Zhejiang University, Xing Hu Zhejiang University, Xin Xia Zhejiang University | ||
12:10 10mTalk | Provable Fairness Repair for Deep Neural Networks Research Papers Jianan Ma Hangzhou Dianzi University, China; Zhejiang University, Hangzhou, China, Jingyi Wang Zhejiang University, Qi Xuan Zhejiang University of Technology; Binjiang Institute of Artificial Intelligence, Zhen Wang Hangzhou Dianzi University, China | ||
12:20 10mTalk | AutoAdapt: On the Application of AutoML for Parameter-Efficient Fine-Tuning of Pre-Trained Code Models Journal-First Amal Akli University of Luxembourg, Maxime Cordy University of Luxembourg, Luxembourg, Mike Papadakis University of Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg | ||