AST 2025
Sat 26 April - Sun 4 May 2025 Ottawa, Ontario, Canada
co-located with ICSE 2025
Tue 29 Apr 2025 11:45 - 12:07 at 211 - Session 4: When and How to Test

Examples in web API specifications can be essential for API testing, API understanding, and even building chat-bots for APIs. Unfortunately, most API specifications lack human written examples. This paper introduces a novel technique for generating examples for web API specifications. We start from in-context learning (ICL): given an API parameter, use a prompt context containing a few examples from other similar API parameters to call a model to generate new examples. However, while ICL tends to generate correct examples, those lack diversity, which is also important for most downstream tasks. Therefore, we extend the technique to iterated-calls ICL (ICICL): use a few different prompt contexts, each containing a few examples, to iteratively call the model with each context. Our intrinsic evaluation demonstrates that ICICL improves both correctness and diversity of generated examples. More importantly, our extrinsic evaluation demonstrates that those generated examples significantly improve the performance of downstream tasks of testing, understanding, and chat-bots for APIs.

Tue 29 Apr

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
Session 4: When and How to TestAST 2025 at 211

Session chair: Mehrdad Saadatmand

11:00
22m
Full-paper
An Adaptive Testing Approach Based on Field Data
AST 2025
Samira Santos da Silva Gran Sasso Science Institute (GSSI), Ricardo Caldas Gran Sasso Science Institute (GSSI), Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy, Antonia Bertolino Gran Sasso Science Institute
Pre-print
11:22
22m
Full-paper
Exceptional Behaviors: How Frequently Are They Tested?
AST 2025
Andre Hora UFMG, Gordon Fraser University of Passau
Pre-print Media Attached
11:45
22m
Full-paper
Improving Examples in Web API Specifications using Iterated-Calls In-Context Learning
AST 2025
Kush Jain Carnegie Mellon University, Kiran Kate IBM Research, Jason Tsay IBM Research, Claire Le Goues Carnegie Mellon University, Martin Hirzel IBM Research
Pre-print
12:07
22m
Full-paper
What Types of Automated Tests do Developers Write?
AST 2025
Marko Ivanković University of Passau, Luka Rimanić Google, Ivan Budiselic Google, Goran Petrovic Google; Universität Passau, Gordon Fraser University of Passau, René Just University of Washington
Pre-print
:
:
:
: