ISEC 2025 (series) / Research Papers /
Test Case Generation for Requirements in Natural Language - An LLM Comparison Study
Fri 21 Feb 2025 12:37 - 13:00 at Senate Hall - Research Papers Session 1: LLM for Software Engineering
The rapid evolution of Large Language Models (LLMs) has unlocked new possibilities for automating tasks throughout the software development lifecycle, including test case generation. This paper presents a comparative analysis of six LLMs in generating test cases for technical requirements written in natural language (English). The study evaluates publicly available general-purpose LLMs, namely BARD, ChatGPT 3.5, Claude, Gemini, ChatGPT 4.0 (Omni), and Llama 3. The generated test cases are validated against a Simulink model created for the corresponding requirements, and the resulting coverage metrics are used for a quantitative comparison of the LLMs.
Fri 21 FebDisplayed time zone: Chennai, Kolkata, Mumbai, New Delhi change
Fri 21 Feb
Displayed time zone: Chennai, Kolkata, Mumbai, New Delhi change
11:30 - 13:00 | |||
11:30 22mResearch paper | Constructing Enterprise Digital Twins by Augmenting LLMs with MDE Research Papers Souvik Barat Tata Consultancy Services Research, Dushyanthi Mulpuru , abhishek Yadav , Himabindu Thogaru , Vinay Kulkarni Tata Consultancy Services Research | ||
11:52 22mResearch paper | REST API Functional Tester Research Papers Diptikalyan Saha IBM Research India, Devika Sondhi IBM Research, Swagatam Haldar , Saurabh Sinha IBM Research | ||
12:15 22mResearch paper | SolGen: Secure Smart Contract Code Generation Using Large Language Models Via Masked Prompting Research Papers Md Tauseef Alam Indian Institute of Technology Patna, Sorbajit Goswami Indian Institute of Technology Patna, Khushi Singh Indian Institute of Technology Patna, Soumyadip Bandyopadhyay industry, Raju Halder Indian Institute of Technology Patna, Abyayananda Maiti | ||
12:37 22mResearch paper | Test Case Generation for Requirements in Natural Language - An LLM Comparison Study Research Papers |