Quality assurance study with mismatched data in sentiment analysis
Considering mismatched data have harmful influence on the quality assurance in sentiment analysis, therefore this paper proposed an effective method to detect these mismatches. This study considered data adequacy and model’s confidence together, and proposed modifications on the surprise adequacy metric to develop methods for mismatch detection. Experiments were implemented on Amazon.com review data. Performances of mismatched data detection and retrained models with deletion of mismatched data were evaluated. The proposed method using Mahalanobis distance and distance-based surprise adequacy was verified feasible and effective to detect the mismatched review comments in the studied Amazon.com review dataset. Moreover, the related study on mismatch detection was illustrated useful and meaningful for AI quality management in sentiment analysis.
Fri 9 DecDisplayed time zone: Osaka, Sapporo, Tokyo change
09:20 - 10:30 | Review and Natural Language ProcessingTechnical Track / ERA - Early Research Achievements at Room1 Chair(s): Renuka Sindhgatta IBM Research AI | ||
09:20 15mPaper | A checklist-based approach to assess the systematicity of the abstracts of reviews self-identifying as systematic reviews ERA - Early Research Achievements | ||
09:35 15mPaper | Preliminary Analysis of Review Method Selection Based on Bandit Algorithms ERA - Early Research Achievements Takuto Kudo Kindai University, Masateru Tsunoda Kindai University, Amjed Tahir Massey University, Kwabena Ebo Bennin Wageningen University and Research, Akito Monden Okayama University, Koji Toda Fukuoka Institute of Technology, Keitaro Nakasai National Institute of Technology, Kagoshima College, Kenichi Matsumoto Nara Institute of Science and Technology | ||
09:50 20mPaper | Reducing Syntactic Complexity for Information Extraction from Japanese Requirement Specifications Technical Track Maiko Onishi Ochanomizu University, Shinpei Ogata Shinshu University, Kozo Okano Shinshu University, Daisuke Bekki Ochanomizu University | ||
10:10 15mPaper | Quality assurance study with mismatched data in sentiment analysis ERA - Early Research Achievements Tinghui Ouyang National Institute of Advanced Industrial Science and Technology, Yoshiki Seo National Institute of Advanced Industrial Science and Technology, Yutaka Oiwa National Institute of Advanced Industrial Science and Technology | ||