An empirical study of test case prioritization on the Linux Kernel
This program is tentative and subject to change.
The Linux kernel is a complex and constantly evolving system, where each code change can impact different components of the system. Regression testing ensures that new changes do not affect existing functionality or introduce new defects. However, due to the complexity of the Linux kernel, maintenance remains challenging. While practices like Continuous Integration (CI) facilitate rapid commits through automated regression testing, each CI process still incurs substantial costs due to the extensive number of test cases. Traditional software testing employs test case prioritization (TCP) techniques to prioritize test cases, thus enabling the early detection of defects. Due to the unique characteristics of the Linux kernel, it remains unclear whether the existing TCP techniques are suitable for its regression testing. In this paper, we present the first empirical study by comparing various TCP techniques in Linux kernel context. Specifically, we examined a total of 17 TCP techniques, including similarity-based, information-retrieval-based, and coverage-based techniques. The experimental results demonstrate that: (1) Similarity-based TCP techniques perform best on the Linux kernel, achieving a mean APFD (Average Percentage of Faults Detected) value of 0.7583 and requiring significantly less time; (2) The majority of TCP techniques show relatively stable performance across multiple commits, where similarity-based TCP techniques are more stable with a maximum decrease of 3.03% and 3.92% in terms of mean and median APFD values, respectively; (3) More than half of the studied techniques are significantly affected by flaky tests, with both mean and median APFD values ranging from -29.9% to -63.5%. This work takes the first look at the adoption of TCP techniques in the Linux kernel, confirming its potential for effective and efficient prioritization.
This program is tentative and subject to change.
Wed 19 NovDisplayed time zone: Seoul change
14:00 - 15:30 | |||
14:00 10mTalk | LLMs for Automated Unit Test Generation and Assessment in Java: The AgoneTest Framework Research Papers Andrea Lops Polytechnic University of Bari, Italy, Fedelucio Narducci Polytechnic University of Bari, Azzurra Ragone University of Bari, Michelantonio Trizio Wideverse, Claudio Bartolini Wideverse s.r.l. | ||
14:10 10mTalk | µOpTime: Statically Reducing the Execution Time of Microbenchmark Suites Using Stability Metrics Journal-First Track Nils Japke TU Berlin & ECDF, Martin Grambow TU Berlin & ECDF, Christoph Laaber Simula Research Laboratory, David Bermbach TU Berlin | ||
14:20 10mTalk | Reference-Based Retrieval-Augmented Unit Test Generation Journal-First Track Zhe Zhang Beihang University, Liu Xingyu Beihang University, Yuanzhang Lin Beihang University, Xiang Gao Beihang University, Hailong Sun Beihang University, Yuan Yuan Beihang University | ||
14:30 10mTalk | Using Active Learning to Train Predictive Mutation Testing with Minimal Data Research Papers Miklos Borsi Karlsruhe Institute of Technology | ||
14:40 10mTalk | Clarifying Semantics of In-Context Examples for Unit Test Generation Research Papers Chen Yang Tianjin University, Lin Yang Tianjin University, Ziqi Wang Tianjin University, Dong Wang Tianjin University, Jianyi Zhou Huawei Cloud Computing Technologies Co., Ltd., Junjie Chen Tianjin University | ||
14:50 10mTalk | An empirical study of test case prioritization on the Linux Kernel Journal-First Track Haichi Wang College of Intelligence and Computing, Tianjin University, Ruiguo Yu College of Intelligence and Computing, Tianjin University, Dong Wang Tianjin University, Yiheng Du College of Intelligence and Computing, Tianjin University, Yingquan Zhao Tianjin University, Junjie Chen Tianjin University, Zan Wang Tianjin University | ||
15:00 10mTalk | Automated Generation of Issue-Reproducing Tests by Combining LLMs and Search-Based Testing Research Papers Konstantinos Kitsios University of Zurich, Marco Castelluccio Mozilla, Alberto Bacchelli University of Zurich Pre-print | ||
15:10 10mTalk | Using Fourier Analysis and Mutant Clustering to Accelerate DNN Mutation Testing Research Papers | ||
15:20 10mTalk | WEST: Specification-Based Test Generation for WebAssembly Research Papers | ||