REACCEPT: Automated Co-evolution of Production and Test Code Based on Dynamic Validation and Large Language Models
Synchronizing production and test code, known as PT co-evolution, is critical for software quality in the software development lifecycle. Existing methods for automatic PT co-evolution either utilize predefined heuristic rules or rely on simple application of machine learning techniques. Due to the limitations of underlying techniques, existing methods either only partially automate PT co-evolution (e.g., only automate obsolete test code identification) or result in low accuracy.
In this paper, we propose REACCEPT, a novel approach that leverages large language models and dynamic validation to fully automate PT co-evolution (i.e., capable of both identifying and updating obsolete test cases). REACCEPT relies on experience-based prompt template generation, dynamic validation, and retrieval-augmented generation techniques to accomplish automated PT co-evolution. To evaluate REACCEPT’s effectiveness, we extensive experiments with a dataset of 537 Java projects and compared REACCEPT’s performance with several state-of-the-art methods. Results show that REACCEPT achieved an update accuracy of 60.16% on correctly identified obsolete test code, surpassing the state-of-the-art technique CEPROT by 90%. This confirms that REACCEPT can effectively assist developers in maintaining test code, improving overall software quality and reducing maintenance effort.
Thu 26 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:00 - 12:30 | Test Automation, Evolution, and API TestingResearch Papers / Tool Demonstrations at Cosmos 3C Chair(s): Alexi Turcotte CISPA | ||
11:00 25mTalk | You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects Research Papers DOI | ||
11:25 25mTalk | Effective REST APIs Testing with Error Message Analysis Research Papers Lixin Xu Nanjing University, China, Huayao Wu Nanjing University, Zhenyu Pan , Tongtong Xu Huawei, Shaohua Wang Central University of Finance and Economics, Xintao Niu Nanjing University, Changhai Nie Nanjing University DOI | ||
11:50 25mTalk | REACCEPT: Automated Co-evolution of Production and Test Code Based on Dynamic Validation and Large Language Models Research Papers Jianlei Chi , Xiaotian Wang Harbin Engineering University, Yuhan Huang Xidian University, Lechen Yu Microsoft, Di Cui Xidian University, Jianguo Sun Xidian University, Jun Sun Singapore Management University DOI | ||
12:15 15mDemonstration | PatchScope – A Modular Tool for Annotating and Analyzing Contributions Tool Demonstrations Jakub Narębski Nicolaus Copernicus University in Toruń, Mikołaj Fejzer Nicolaus Copernicus University in Toruń, Krzysztof Stencel University of Warsaw, Piotr Przymus Nicolaus Copernicus University in Toruń, Poland Link to publication DOI |
Cosmos 3C is the third room in the Cosmos 3 wing.
When facing the main Cosmos Hall, access to the Cosmos 3 wing is on the left, close to the stairs. The area is accessed through a large door with the number “3”, which will stay open during the event.