Dynamic Mutation Scheduling: Highly Parallel, Efficient Evaluation of Mutations for Rust Programs through Program Splitting

Mutation analysis has been considered prohibitively expensive for large software projects due to its high computational cost. Even with efficient mutant schemata approaches, the evaluation of the constituent mutants still takes considerable time, and does not scale well as the size of the test suite and the number of mutations increases. Classical mutation analysis systems use processes to orchestrate the evaluation of individual mutants, which introduces large overheads. Our novel Dynamic Mutation Scheduling approach uses threads to evaluate mutant test cases, and further increases utilization of modern computer systems by running multiple, compatible mutations in parallel in the same process, scheduling in new mutations and their test cases as capacity becomes available. Mutations are compatible with regards to parallel evaluation if they appear in distinct parts of the program. Our approach both reduces the overheads of mutant test case evaluation, and increases the amount of parallelized work, resulting in reduced mutation evaluation runtimes. We implement our approach by extending mutest-rs, the state-of-the- art mutation analysis tool for the Rust programming language. We perform an extensive empirical evaluation (69,974 CPU minutes) of our novel Dynamic Mutation Scheduling approach on 25 top subject Rust programs totaling 773,655 SLoC, comparing it to a basic thread-based mutation evaluation approach, and to state-of-the-art Static Mutation Batching; a more simplified approach to parallel mutation evaluation. We find that Dynamic Mutation Scheduling effectively reduces wasted time and mutation evaluation runtimes over Static Mutation Batching, by up to 75.6%, while also reducing exhaustive mutation analysis runtimes, by up to 25.2%.