Knowledge Matters: Injecting Project and Testing Knowledge into LLM-based Unit Test Generation
Automated unit test generation using large language models (LLMs) holds great promise but often struggles with generating tests that are both correct and maintainable in real-world projects. This paper presents KTester, a novel framework that integrates project-specific knowledge and testing domain knowledge to enhance LLM-based test generation. Our approach first extracts project structure and usage knowledge through static analysis, which provides rich context for the model. It then employs a testing-domain-knowledge-guided separation of test case design and test method generation, combined with a multi-perspective prompting strategy that guides the LLM to consider diverse testing heuristics. The generated tests follow structured templates, improving clarity and maintainability. We evaluate KTester on multiple open-source projects, comparing it against state-of-the-art LLM-based baselines using automatic correctness and coverage metrics, as well as a human study assessing readability and maintainability. Results demonstrate that KTester significantly outperforms existing methods across all metrics, producing tests with clearer intent and higher practical value.
Thu 16 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | AI for Software Engineering 14Research Track at Asia IV Chair(s): Reyhaneh Jabbarvand University of Illinois at Urbana-Champaign | ||
14:00 15mTalk | Knowledge Matters: Injecting Project and Testing Knowledge into LLM-based Unit Test Generation Research Track Anji Li School of Software Engineering, Sun Yat-sen University, Mingwei Liu Sun Yat-Sen University, Zhenxi Chen Sun Yat-Sen University, Zheng Pei Sun Yat-Sen University, Zike Li Sun Yat-Sen University, Dekun Dai Sun Yat-Sen University, Yanlin Wang Sun Yat-sen University, Zibin Zheng Sun Yat-sen University DOI Pre-print Media Attached | ||
14:15 15mTalk | Issue2Test: Generating Reproducing Test Cases from Issue Reports Research Track Noor Nashid University of British Columbia, Islem BOUZENIA CISPA Helmholtz Center for Information Security, Michael Pradel CISPA Helmholtz Center for Information Security, Ali Mesbah University of British Columbia | ||
14:30 15mTalk | RBCTest: Leveraging LLMs to Mine and Verify Oracles of API Response Bodies for RESTful API Testing Research Track Hieu Huynh University of Melbourne, Quoc-Tri Le Katalon LLC, Tu Nguyen University of Science, VNU-HCM, Viet Nguyen University of Science, VNU-HCM, Vu Nguyen University of Science, VNU-HCM; Katalon LLC., Tien N. Nguyen University of Texas at Dallas | ||
14:45 15mTalk | Measuring the Influence of Incorrect Code on Test Generation Research Track Dong Huang The University of Hong Kong, Jie M. Zhang King's College London, Mark Harman Meta Platforms, Inc. and UCL, Mingzhe Du National University of Singapore, Heming Cui University of Hong Kong | ||
15:00 15mTalk | Retrieval-Augmented Test Generation: How Far Are We? Research Track Jiho Shin Queen's University, Nima Shiri Harzevili York University, Reem Aleithan York University, Canada, Hadi Hemmati York University, Song Wang York University Pre-print | ||
15:15 15mTalk | SAINT: Service-level Integration Test Generation with Program Analysis and LLM-based Agents Research Track Rangeet Pan IBM Research, Raju Pavuluri IBM T.J. Watson Research Center, Ruikai Huang Georgia Institute of Technology, Tyler Stennett Georgia Institute of Technology, Rahul Krishna IBM Research, Alessandro Orso University of Georgia, USA, Saurabh Sinha IBM Research | ||