ICST 2024
Mon 27 - Fri 31 May 2024 Canada
Thu 30 May 2024 11:00 - 11:20 at Room 2 & 3 - Mutation Testing and Test Prioritizaiton Chair(s): Facundo Molina

This paper reports on an industrial success story of using mutation testing for evaluating and improving test suites for smart contracts. ERCx is an industry scale, hand-written test suite and framework for testing smart contracts,developed by Runtime Verification. Hand-written tests often overlook edge cases. To improve test suites, mutation testing employs a systematic approach of introducing small, syntactic changes to a program. These changes, called “mutants”, act as proxies for real faults. Mutants that are not detected by the test suite indicate potential weaknesses and can be used as test goals. We show that mutation testing finds test cases that were missing from ERCx, even after substantial time and effort investment from developers. Based on “live mutants”, we found five missing tests. One of them was deemed critical by the ERCx developers indicating that all contracts must pass this test and all five are now part of the ERCx test suite. The generated mutants revealed interesting correlations between the tests. Using this correlation model, we developed a mechanism for test minimization without sacrificing coverage. The minimized test suite is equally effective in finding bugs in real-world programs while being faster to run.

Thu 30 May

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:40
Mutation Testing and Test PrioritizaitonResearch Papers / Journal-First Papers / Industry at Room 2 & 3
Chair(s): Facundo Molina IMDEA Software Institute
11:00
20m
Industry talk
Towards Mutation-guided Test Suites for Smart Contracts
Industry
Pre-print
11:20
20m
Research paper
On the Coupling between Vulnerabilities and LLM-generated Mutants: A Study on Vul4J dataset
Research Papers
Aayush Garg Luxembourg Institute of Science and Technology, Renzo Degiovanni Luxembourg Institute of Science and Technology, Mike Papadakis University of Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg
Pre-print
11:40
20m
Long-paper
Mutation Testing Optimisations using the Clang Front-end
Journal-First Papers
Sten Vercammen , Serge Demeyer University of Antwerp; Flanders Make, Markus Borg CodeScene, Niklas Pettersson , Görel Hedin Lund University
12:00
20m
Research paper
MACS: Multi-agent Adversarial Reinforcement Learning for Finding Diverse Critical Driving Scenarios
Research Papers
Shuting Kang University of Chinese Academy of Sciences, Qian Dong Institute of Software Chinese Academy of Sciences, Yunzhi Xue Institute of Software Chinese Academy of Sciences, Yanjun Wu Institute of Software, Chinese Academy of Sciences
12:20
20m
Long-paper
Lessons learned from replicating a study on information-retrieval-based test case prioritization
Journal-First Papers
Nasir Mehmood Minhas Mälardalen University, Mohsin Irshad , Kai Petersen University of Applied Sciences Flensburg, Germany / Blekinge Institute of Technology, Sweden, Jürgen Börstler Blekinge Institute of Technology