Automated Evolutionary Hyperparameter Tuning for NLP-Based Test Case Generation
Automated generation of executable test suites from natural-language requirements remains challenging due to linguistic ambiguity and sensitivity of generative models to decoding and training hyperparameters. This paper introduces a hierarchical, multi-level evolutionary framework that treats model hyperparameters and decod- ing strategies as upper-level decision variables and em- ploys lower-level fitnesses that directly measure test-quality objectives (structural coverage, semantic diversity, redun- dancy, and runtime efficiency). The approach integrates retrieval-augmented grounding, surrogate-assisted preselec- tion, lightweight LoRA adaptation and optional HIL evalu- ation. Empirical evaluation on PURE, PROMISE exp and FR NFR benchmarks (repeated runs, n = 10; paired two- sided t-tests, α = 0.05) shows consistent gains: on PURE mean code coverage reaches 82.4% (vs. 75.1% for Bayesian optimisation and 68.9% for random search) with 145 unique scenarios and modest runtime overhead (≈58.3 s, ≈6% above Bayesian). Ablations confirm component effects (e.g., removing diversity reduces unique scenarios ≈18%; disabling the surrogate increases wall-clock ≈42%; dis- abling RAG drops grounded consistency ≈12%). Results indicate that co-optimising hyperparameters for explicit test-quality metrics, together with grounding and realistic execution, yields more useful, executable test suites. Future work will explore adaptive objective weighting, transfer warm-starts and probabilistic surrogates.
Sun 16 NovDisplayed time zone: Seoul change
14:00 - 15:30 | |||
14:00 30mKeynote | Keynote Speech Intelligent SE 2025 | ||
14:30 15mTalk | Leveraging Large Language Models for Use Case Model Generation from Software Requirements Intelligent SE 2025 Tobias Eisenreich Technical University of Munich, Nicholas Friedlaender Technical University of Munich (TUM), Stefan Wagner Technical University of Munich | ||
14:45 15mTalk | AI for Requirements Engineering: Industry adoption and Practitioner perspectives Intelligent SE 2025 Lekshmi Murali Rani Chalmers University of Technology and University of Gothenburg, Sweden, Richard Berntsson Svensson Chalmers University of Technology & University of Gothenburg, Robert Feldt Chalmers | University of Gothenburg Pre-print | ||
15:00 15mTalk | LLMs Choose the Right Stack: From Patterns to Tools Intelligent SE 2025 Sebastian Copei Fraunhofer IEE, Oliver Hohlfeld University of Kassel, Jens Kosiol Philipps-Universität Marburg, Aleksandar Ristoski Fraunhofer IEE | ||
15:15 15mTalk | Automated Evolutionary Hyperparameter Tuning for NLP-Based Test Case Generation Intelligent SE 2025 Ivan Malashin Bauman Moscow State Technical University, Igor Masich Bauman Moscow State Technical University, Sergei Kurashkin Bauman Moscow State Technical University, Andrei Gantimurov Bauman Moscow State Technical University, Aleksei Borodulin Bauman Moscow State Technical University, Vadim Tynchenko Bauman Moscow State Technical University, Vladimir Nelyub Bauman Moscow State Technical University | ||