STRUT: Structured Seed Case Guided Unit Test Generation for C Programs using LLMs
Unit testing plays a crucial role in bug detection and ensuring software correctness. It helps developers identify errors early in development, thereby reducing software defects. In recent years, large language models (LLMs) have demonstrated significant potential in automating unit test generation. However, using LLMs to generate unit tests faces many challenges. 1) The execution pass rate of the test cases generated by LLMs is low. 2) The test case coverage is inadequate, making it challenging to detect potential risks in the code. 3) Current research methods primarily focus on languages such as Java and Python, while studies on C programming are scarce, despite its importance in the real world. To address these challenges, we propose STRUT, a novel unit test generation method. STRUT utilizes structured test cases as a bridge between complex programming languages and LLMs. Instead of directly generating test code, STRUT guides LLMs to produce structured test cases, thereby alleviating the limitations of LLMs when generating code for programming languages with complex features. First, STRUT analyzes the context of focal methods and constructs structured seed test cases for them. These seed test cases then guide LLMs to generate a set of structured test cases. Subsequently, a rule-based approach is employed to convert the structured set of test cases into executable test code. We conducted a comprehensive evaluation of STRUT, which achieved an impressive execution pass rate of 96.01%, along with 77.67% line coverage and 63.60% branch coverage. This performance significantly surpasses that of the LLMs-based baseline methods and the symbolic execution tool SunwiseAUnit. These results highlight STRUT’s superior capability in generating high-quality unit test cases by leveraging the strengths of LLMs while addressing their inherent limitations.
Fri 27 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:00 - 12:15 | |||
11:00 25mTalk | KEENHash: Hashing Programs into Function-aware Embeddings for Large-scale Binary Code Similarity Analysis Research Papers Zhijie Liu ShanghaiTech University, China, Qiyi Tang Tencent Security Keen Lab, Sen Nie Tencent Security Keen Lab, Shi Wu Tencent Security Keen Lab, Liangfeng Zhang School of Information Science and Technology, ShanghaiTech University, Yutian Tang University of Glasgow, United Kingdom DOI | ||
11:25 25mTalk | Porting Software Libraries to OpenHarmony: Transitioning from TypeScript or JavaScript to ArkTS Research Papers Bo Zhou Northeastern University, Jiaqi Shi Northeastern University, Ying Wang Northeastern University, Li Li Beihang University, Li Tsz On The Hong Kong University of Science and Technology, Hai Yu Northeastern University, China, Zhiliang Zhu Northeastern University, China DOI | ||
11:50 25mTalk | STRUT: Structured Seed Case Guided Unit Test Generation for C Programs using LLMs Research Papers Jinwei Liu Xidian University, Chao Li Beijing Institute of Control Engineering; Beijing Sunwise Information Technology, Rui Chen Beijing Institute of Control Engineering; Beijing Sunwise Information Technology, Shaofeng Li Xidian University, Bin Gu Beijing Institute of Control Engineering, Mengfei Yang China Academy of Space Technology DOI | ||
Cosmos 3C is the third room in the Cosmos 3 wing.
When facing the main Cosmos Hall, access to the Cosmos 3 wing is on the left, close to the stairs. The area is accessed through a large door with the number “3”, which will stay open during the event.