Improving Examples in Web API Specifications using Iterated-Calls In-Context Learning
Examples in web API specifications can be essential for API testing, API understanding, and even building chat-bots for APIs. Unfortunately, most API specifications lack human written examples. This paper introduces a novel technique for generating examples for web API specifications. We start from in-context learning (ICL): given an API parameter, use a prompt context containing a few examples from other similar API parameters to call a model to generate new examples. However, while ICL tends to generate correct examples, those lack diversity, which is also important for most downstream tasks. Therefore, we extend the technique to iterated-calls ICL (ICICL): use a few different prompt contexts, each containing a few examples, to iteratively call the model with each context. Our intrinsic evaluation demonstrates that ICICL improves both correctness and diversity of generated examples. More importantly, our extrinsic evaluation demonstrates that those generated examples significantly improve the performance of downstream tasks of testing, understanding, and chat-bots for APIs.
Tue 29 AprDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | |||
11:00 22mFull-paper | An Adaptive Testing Approach Based on Field Data AST 2025 Samira Santos da Silva Gran Sasso Science Institute (GSSI), Ricardo Caldas Gran Sasso Science Institute (GSSI), Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy, Antonia Bertolino Gran Sasso Science Institute Pre-print | ||
11:22 22mFull-paper | Exceptional Behaviors: How Frequently Are They Tested? AST 2025 Pre-print Media Attached | ||
11:45 22mFull-paper | Improving Examples in Web API Specifications using Iterated-Calls In-Context Learning AST 2025 Kush Jain Carnegie Mellon University, Kiran Kate IBM Research, Jason Tsay IBM Research, Claire Le Goues Carnegie Mellon University, Martin Hirzel IBM Research Pre-print | ||
12:07 22mFull-paper | What Types of Automated Tests do Developers Write? AST 2025 Marko Ivanković University of Passau, Luka Rimanić Google, Ivan Budiselic Google, Goran Petrovic Google; Universität Passau, Gordon Fraser University of Passau, René Just University of Washington Pre-print |