Enriching automatic test case generation by extracting relevant test inputs from bug reports
This program is tentative and subject to change.
The quality of software is closely tied to the effectiveness of the tests it undergoes. Manual test writing, though crucial for bug detection, is time-consuming, which has driven significant research into automated test case generation. However, current methods often struggle to generate relevant inputs, limiting the effectiveness of the tests produced. To address this, we introduce BRMiner, a novel approach that leverages Large Language Models (LLMs) in combination with traditional techniques to extract relevant inputs from bug reports, thereby enhancing automated test generation tools. In this study, we evaluate BRMiner using the Defects4J benchmark and test generation tools such as EvoSuite and Randoop. Our results demonstrate that BRMiner achieves a Relevant Input Rate (RIR) of 60.03% and a Relevant Input Extraction Accuracy Rate (RIEAR) of 31.71%, significantly outperforming methods that rely on LLMs alone. The integration of BRMiner’s input enhances EvoSuite ability to generate more effective test, leading to increased code coverage, with gains observed in branch, instruction, method, and line coverage across multiple projects. Furthermore, BRMiner facilitated the detection of 58 unique bugs, including those that were missed by traditional baseline approaches. Overall, BRMiner’s combination of LLM filtering with traditional input extraction techniques significantly improves the relevance and effectiveness of automated test generation, advancing the detection of bugs and enhancing code coverage, thereby contributing to higher-quality software development.
This program is tentative and subject to change.
Wed 10 SepDisplayed time zone: Auckland, Wellington change
13:30 - 15:00 | Session 4 - Testing 1Research Papers Track / Registered Reports / Journal First Track / NIER Track / Industry Track / Tool Demonstration Track at Room TBD2 | ||
13:30 15m | Performance Testing in Open-Source Web Projects: Adoption, Maintenance, and a Change Taxonomy Research Papers Track Sergio Di Meglio Università degli Studi di Napoli Federico II, Luigi Libero Lucio Starace Università degli Studi di Napoli Federico II, Valeria Pontillo Gran Sasso Science Institute, Ruben Opdebeeck Vrije Universiteit Brussel, Coen De Roover Vrije Universiteit Brussel, Sergio Di Martino Università degli Studi di Napoli Federico II Pre-print | ||
13:45 15m | Harnessing LLMs for Document-Guided Fuzzing of OpenCV Library Research Papers Track Bin Duan The University of Queensland, Tarek Mahmud Texas State University, Meiru Che Central Queensland University, Yan Yan University of Illinois Chicago, Naipeng Dong The University of Queensland, Australia, Dan Dongseong Kim The University of Queensland, Guowei Yang University of Queensland | ||
14:00 10m | XTestGen: Natural Language to Maintainable E2E Test Scripts with LLMs Tool Demonstration Track | ||
14:10 10m | Towards Effective Lightweight Test Oracles for Automated Multi-Fault Program Repair NIER Track Omar I. Al-Bataineh Gran Sasso Science Institute (GSSI) | ||
14:20 15m | Testing Is Not Boring: Characterizing Challenge in Software Testing Tasks Industry Track Davi Gama Hardman CESAR - Recife Center for Advanced Studies and Systems, César França Federal Rural University of Pernambuco (UFRPE), Brody Stuart-Verner University of Calgary, Ronnie de Souza Santos University of Calgary | ||
14:35 15m | Enriching automatic test case generation by extracting relevant test inputs from bug reports Journal First Track Wendkuuni Arzouma Marc Christian OUEDRAOGO University of Luxembourg, Laura Plein CISPA Helmholtz Center for Information Security, Abdoul Kader Kaboré University of Luxembourg, Andrew Habib ABB Corporate Research, Germany, Jacques Klein University of Luxembourg, David Lo Singapore Management University, Tegawendé F. Bissyandé University of Luxembourg | ||
14:50 10m | An Empirical Study of Complexity, Heterogeneity, and Compliance of GitHub Actions Workflows Registered Reports Edward Abrokwah Department of Computer Science, Trent University, Peterborough, Canada, Taher A. Ghaleb Trent University Pre-print |