Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia
Mon 15 May 2023 14:05 - 14:25 at Meeting Room 209 - Session 2

MLOps, techniques to solve operational issues in machine learning, have been attracting attention in recent years. In order to continuously improve operational models, it is essential to identify the causes of mispredictions, such as “model over-fitting” and “outliers for training data”, and take appropriate countermeasures. However, misprediction analysis is currently a time-consuming process performed manually by data scientists. To automatically identify the causes of mispredictions, we propose a flowchart-structured analysis method (called AIEDF) that logically integrates the results of a comprehensive analysis of data and models in a form we can understand, i.e., AIEDF is comprehensive and explainable. In addition, AIEDF has flexibility for implementation, and our implementation of AIEDF is model-agnostic and applicable to models including gradient-boosting trees and neural networks. We demonstrated through experiments with synthetic and real data that AIEDF identifies causes with high accuracy and provides valuable insights for model improvement.

Mon 15 May

Displayed time zone: Hobart change

13:45 - 15:15
13:45
20m
Talk
Metamorphic Testing of Machine Translation Models using Back Translation
DeepTest
Wentao Gao University of Melbourne, Jiayuan He RMIT University, Van-Thuan Pham Monash University
14:05
20m
Talk
A Method of Identifying Causes of Prediction Errors to Accelerate MLOps
DeepTest
Keita Sakuma NEC Corporation, Ryuta Matsuno NEC Corporation, Yoshio Kameda NEC Corporation
14:25
20m
Talk
DeepSHAP Summary for Adversarial Example Detection
DeepTest
Yi-Ching Lin National Chengchi University, Fang Yu National Chengchi University
14:45
20m
Talk
DeepPatch: A Patching-Based Method for Repairing Deep Neural Networks
DeepTest
Hao Bu Peking University, Meng Sun Peking University