Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia
Mon 15 May 2023 13:45 - 14:05 at Meeting Room 209 - Session 2

Machine translation software has been widely adopted in recent years. The recent advance in deep learning research has massively improved the accuracy and fluency of the translated output. However, incorrect translations may still occur, which cause misunderstandings, and even more detrimental consequences when applying these systems for crucial applications, such as translating legal and medical documents. This calls for methods that can test the correctness of machine translation software efficiently and effectively. In this paper, we propose a method, which uses back-translation as a reference for machine translation testing, minimizing the knowledge and use of the NLP tools in the target language, so that the same workflow can be applied to test systems translating English to multiple languages. We build a metamorphic testing method using our proposed concept called contextual referentially transparent input (CRTI). A CRTI is a piece of text that should have a similar meaning under a certain context in any given language. Our method detects inconsistency between a CRTI in the original sentence and the back-translation to report translation errors. To evaluate our method, we translate 200 sentences using Google Translate. Our method reports 57 suspicious issues with a precision of 74% in Chinese translation and 22 suspicious issues with a precision of 82% in Vietnamese translation.

Mon 15 May

Displayed time zone: Hobart change

13:45 - 15:15
13:45
20m
Talk
Metamorphic Testing of Machine Translation Models using Back Translation
DeepTest
Wentao Gao University of Melbourne, Jiayuan He RMIT University, Van-Thuan Pham Monash University
14:05
20m
Talk
A Method of Identifying Causes of Prediction Errors to Accelerate MLOps
DeepTest
Keita Sakuma NEC Corporation, Ryuta Matsuno NEC Corporation, Yoshio Kameda NEC Corporation
14:25
20m
Talk
DeepSHAP Summary for Adversarial Example Detection
DeepTest
Yi-Ching Lin National Chengchi University, Fang Yu National Chengchi University
14:45
20m
Talk
DeepPatch: A Patching-Based Method for Repairing Deep Neural Networks
DeepTest
Hao Bu Peking University, Meng Sun Peking University