Understanding on the Edge: LLM-generated Boundary Test Explanations
Boundary value analysis and testing (BVT) is fundamental in software quality assurance because faults tend to cluster at input extremes, yet testers often struggle to understand and justify why certain input-output pairs represent meaningful behavioral boundaries. Large Language Models (LLMs) could help by producing natural-language rationales, but their value for BVT has not been empirically assessed. We therefore conducted an exploratory study on LLM-generated boundary explanations: in a survey, twenty-seven software professionals rated GPT-4.1 explanations for twenty boundary pairs on clarity, correctness, completeness, and perceived usefulness, and six of them elaborated in follow-up interviews. Overall, 63.5% of all ratings were positive (4–5 on a five-point Likert scale) compared to 17% negative (1–2), indicating general agreement but also variability in perceptions. Participants favored explanations that followed a clear structure, cited authoritative sources, and adapted their depth to the reader’s expertise; they also stressed the need for actionable examples to support debugging and documentation. From these insights, we distilled a seven-item requirement checklist that defines concrete design criteria for future LLM-based boundary explanation tools. The results suggest that, with further refinement, LLM-based tools can support testing workflows by making boundary explanations more actionable and trustworthy.
Mon 13 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | Session 3: Test Case Generation and FuzzingAST 2026 at Oceania VI Chair(s): Cristian Augusto University of Oviedo | ||
14:00 30mTalk | Improving Deep Learning Library Testing with Machine Learning AST 2026 Facundo Molina Complutense University of Madrid, M M Abid Naziri North Carolina State University, Feiran Qin North Carolina State University, Alessandra Gorla IMDEA Software Institute, Marcelo d'Amorim North Carolina State University | ||
14:30 30mTalk | Understanding on the Edge: LLM-generated Boundary Test Explanations AST 2026 Sabina Akbarova Chalmers University of Technology, Felix Dobslaw Mid Sweden University, Robert Feldt Chalmers | University of Gothenburg Pre-print | ||
15:00 30mTalk | Search-Based Fuzzing For RESTful APIs That Use MongoDB AST 2026 Hernan Ghianni University of Buenos Aires, Man Zhang Beihang University, China, Juan Pablo Galeotti University of Buenos Aires, Andrea Arcuri Kristiania University College and Oslo Metropolitan University | ||