AST 2023
Mon 15 - Tue 16 May 2023 Melbourne, Australia
co-located with ICSE 2023
Mon 15 May 2023 12:07 - 12:30 at Meeting Room 107 - Faults, AI and Tools

Recently many mutation testing tools have been proposed that rely on bug-fix patterns and natural language models trained on large code corpus. As these tools operate fundamentally differently from the grammar-based traditional approaches, a question arises of how these tools compare in terms of 1) fault detection and 2) cost-effectiveness. Simultaneously, mutation testing research proposes mutant selection approaches based on machine learning to mitigate its application cost. This raises another question: How do the existing mutation testing tools compare when guided by mutant selection approaches? To answer these questions, we compare four existing tools – μBERT (uses pre-trained language model for fault seeding), IBIR (relies on inverted fix-patterns), DeepMutation (generates mutants by employing Neural Machine Translation) and PIT (ap- plies standard grammar-based rules) in terms of fault detection capability and cost-effectiveness, in conjunction with standard and deep learning based mutant selection strategies. Our results show that IBIR has the highest fault detection capability among the four tools; however, it is not the most cost-effective when considering different selection strategies. On the other hand, μBERT having a relatively lower fault detection capability, is the most cost-effective among the four tools. Our results also indicate that comparing mutation testing tools when using deep learning-based mutant selection strategies can lead to different conclusions than the standard mutant selection. For instance, our results demonstrate that combining μBERT with deep learning-based mutant selection yields 12% higher fault detection than the considered tools.

pre-print (AST2023.pdf)468KiB

Mon 15 May

Displayed time zone: Hobart change

11:00 - 12:30
Faults, AI and ToolsAST 2023 at Meeting Room 107
11:00
22m
Talk
An Method of Intelligent Duplicate Bug Report Detection Based on Technical Term Extraction
AST 2023
Xiaoxue Wu Yangzhou University, Wenjing Shan Yangzhou University, Wei Zheng Northwestern Polytechnical University, Zhiguo Chen Northwestern Polytechnical University, Tao Ren Yangzhou University, Xiaobing Sun Yangzhou University
11:22
22m
Talk
A Reinforcement Learning Approach to Generate Test Cases for Web Applications
AST 2023
Xiaoning Chang Institute of Software, Chinese Academy of Sciences, Zheheng Liang Joint Laboratory on Cyberspace Security of China Southern Power Grid, Yifei Zhang State Key Lab of Computer Sciences, Institute of Software, Chinese Academy of Sciences, Lei Cui Joint Laboratory on Cyberspace Security of China Southern Power Grid, Zhenyue Long , Guoquan Wu Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Nanjing College; China Southern Power Grid, Yu Gao Institute of Software, Chinese Academy of Sciences, China, Wei Chen Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Nanjing College, Jun Wei Institute of Software at Chinese Academy of Sciences; University of Chinese Academy of Sciences; University of Chinese Academy of Sciences Chongqing School, Tao Huang Institute of Software Chinese Academy of Sciences
11:45
22m
Talk
Cross-Project setting using Deep learning Architectures in Just-In-Time Software Fault Prediction: An Investigation
AST 2023
Sushant Kumar Pandey Chalmers and University of Gothenburg, Anil Kumar Tripathi Indian Institute of Technology (BHU), Varanasi
12:07
22m
Talk
On Comparing Mutation Testing Tools through Learning-based Mutant SelectionBest  Paper Award
AST 2023
Milos Ojdanic University of Luxembourg, Ahmed Khanfir University of Luxembourg, Aayush Garg University of Luxembourg, Luxembourg, Renzo Degiovanni SnT, University of Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg
File Attached