APITestGenie: Generating Web API Tests from Requirements and API Specifications with LLMs
Modern software systems rely heavily on Web APIs, yet creating meaningful and executable test scripts remains a largely manual, time-consuming, and error-prone task. In this paper, we present \textit{APITestGenie}, a novel tool that leverages Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and prompt engineering to automatically generate API integration tests directly from business requirements and OpenAPI specifications. We evaluated \textit{APITestGenie} on 10 real-world APIs, including 8 APIs comprising circa 1,000 live endpoints from an industrial partner in the automotive domain. The tool was able to generate syntactically and semantically valid test scripts for 89% of the business requirements under test after at most three attempts. Notably, some generated tests revealed previously unknown defects in the APIs, including integration issues between endpoints. Statistical analysis identified API complexity and the level of detail in business requirements as primary factors influencing success rates, with the level of detail in API documentation also affecting outcomes. Feedback from industry practitioners confirmed strong interest in adoption, substantially reducing the manual effort in writing acceptance tests, and improving the alignment between tests and business requirements.
Tue 14 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | |||
14:00 30mTalk | L-SCALE: Locality-Sensitive Coverage for Automata LEarning AST 2026 Mark Giraud Fraunhofer IOSB, Bastian Engel Fraunhofer IOSB, Lea Nasarek Fraunhofer IOSB, Yannis Storrer Fraunhofer IOSB, Philipp Takacs Fraunhofer IOSB, Leon Philipp Wittemund Fraunhofer IOSB | ||
14:30 30mTalk | APITestGenie: Generating Web API Tests from Requirements and API Specifications with LLMs AST 2026 André Pereira Deloitte and Faculty of Engineering, University of Porto, Bruno Lima LIACC, Faculty of Engineering, University of Porto, João Pascoal Faria Faculty of Engineering, University of Porto and INESC TEC Pre-print | ||
15:00 30mTalk | Software Testing Education in the LLM Era: Insights and Emerging Theory AST 2026 Samhitha Dwarakanath Pennsylvania State University, Nathalia Nascimento Pennsylvania State University, Everton Guimaraes Pennsylvania State University | ||