AST 2025
Sat 26 April - Sun 4 May 2025 Ottawa, Ontario, Canada
co-located with ICSE 2025
Mon 28 Apr 2025 11:00 - 11:30 at 211 - Session 1: LLM for Testing

Large language model (LLM)-powered assistants are increasingly used for generating program code and unit tests, but their application in acceptance testing remains underexplored. To help address this gap, this paper explores the use of LLMs for generating executable acceptance tests for web applications through a two-step process: (i) generating acceptance test scenarios in natural language (in Gherkin) from user stories, and (ii) converting these scenarios into executable test scripts (in Cypress), knowing the HTML code of the pages under test. This two-step approach supports acceptance test-driven development, enhances tester control, and improves test quality. The two steps were implemented in the AutoUAT and Test Flow tools, respectively, powered by GPT-4 Turbo, and integrated into a partner company’s workflow and evaluated on real-world projects. The users found the acceptance test scenarios generated by AutoUAT helpful 95% of the time, even revealing previously overlooked cases. Regarding Test Flow, 92% of the acceptance test cases generated by Test Flow were considered helpful: 60% were usable as generated, 8% required minor fixes, and 24% needed to be regenerated with additional inputs; the remaining 8% were discarded due to major issues. These results suggest that LLMs can, in fact, help improve the acceptance test process, with appropriate tooling and supervision.

Mon 28 Apr

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
Session 1: LLM for TestingAST 2025 at 211
11:00
30m
Full-paper
Acceptance Test Generation with Large Language Models: An Industrial Case Study
AST 2025
Margarida Ferreira University of Porto and Critical TechWorks, Luís Viegas University of Porto and Critical TechWorks, João Pascoal Faria Faculty of Engineering, University of Porto and INESC TEC, Bruno Lima Faculty of Engineering of the University of Porto & LIACC
Pre-print
11:30
30m
Full-paper
AsserT5: Test Assertion Generation Using a Fine-Tuned Code Language Model
AST 2025
Severin Primbs University of Passau, Benedikt Fein University of Passau, Gordon Fraser University of Passau
Pre-print
12:00
30m
Full-paper
Simulink Mutation Testing using CodeBERT
AST 2025
Jingfan Zhang University of Ottawa, Delaram Ghobari University of Ottawa, Mehrdad Sabetzadeh University of Ottawa, Shiva Nejati University of Ottawa
Pre-print