A Method of Identifying Causes of Prediction Errors to Accelerate MLOps
MLOps, techniques to solve operational issues in machine learning, have been attracting attention in recent years. In order to continuously improve operational models, it is essential to identify the causes of mispredictions, such as “model over-fitting” and “outliers for training data”, and take appropriate countermeasures. However, misprediction analysis is currently a time-consuming process performed manually by data scientists. To automatically identify the causes of mispredictions, we propose a flowchart-structured analysis method (called AIEDF) that logically integrates the results of a comprehensive analysis of data and models in a form we can understand, i.e., AIEDF is comprehensive and explainable. In addition, AIEDF has flexibility for implementation, and our implementation of AIEDF is model-agnostic and applicable to models including gradient-boosting trees and neural networks. We demonstrated through experiments with synthetic and real data that AIEDF identifies causes with high accuracy and provides valuable insights for model improvement.
Mon 15 MayDisplayed time zone: Hobart change
13:45 - 15:15 | |||
13:45 20mTalk | Metamorphic Testing of Machine Translation Models using Back Translation DeepTest | ||
14:05 20mTalk | A Method of Identifying Causes of Prediction Errors to Accelerate MLOps DeepTest | ||
14:25 20mTalk | DeepSHAP Summary for Adversarial Example Detection DeepTest | ||
14:45 20mTalk | DeepPatch: A Patching-Based Method for Repairing Deep Neural Networks DeepTest |