REST APIs are essential in building modern enterprise systems, while effectively examining their behaviors remains challenging due to the difficulty in inferring constraints from the specifications. To generate valid test inputs for REST APIs, existing approaches are typically feedback-driven, leveraging HTTP status codes received to guide further test input generation. However, these approaches overlook the potentially valuable information described in error messages accompanying HTTP status codes, leading to inefficiencies in exploring the input space of REST APIs. In this paper, we propose EmRest, a black-box testing approach that leverages error message analysis to enhance both valid and exceptional test input generation for REST APIs. For each operation under test, EmRest first identifies all possible value assignment strategies for each of its input parameters. It then repeatedly applies combinatorial testing to sample test inputs based on these strategies, and statistically analyzes the error messages (of 400-range status code) received to infer and exclude invalid combinations of value assignment strategies (i.e., constraints of the input space). Additionally, EmRest seeks to mutate valid value assignment strategies that are finally identified to generate test inputs for exceptional testing. The error messages (of 500-range status code) received are categorized to identify bug-prone operations, for which more testing resources are allocated. Our experimental results on 16 real-world REST APIs demonstrates the effectiveness of EmRest. It achieves higher operation coverage than state-of-the-art approaches in 50% of APIs, and detects 226 unique bugs that cannot be found by the other approaches.
Thu 26 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:00 - 12:30 | Test Automation, Evolution, and API TestingResearch Papers / Tool Demonstrations at Cosmos 3C Chair(s): Alexi Turcotte CISPA | ||
11:00 25mTalk | You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects Research Papers DOI | ||
11:25 25mTalk | Effective REST APIs Testing with Error Message Analysis Research Papers Lixin Xu Nanjing University, China, Huayao Wu Nanjing University, Zhenyu Pan , Tongtong Xu Huawei, Shaohua Wang Central University of Finance and Economics, Xintao Niu Nanjing University, Changhai Nie Nanjing University DOI | ||
11:50 25mTalk | REACCEPT: Automated Co-evolution of Production and Test Code Based on Dynamic Validation and Large Language Models Research Papers Jianlei Chi , Xiaotian Wang Harbin Engineering University, Yuhan Huang Xidian University, Lechen Yu Microsoft, Di Cui Xidian University, Jianguo Sun Xidian University, Jun Sun Singapore Management University DOI | ||
12:15 15mDemonstration | PatchScope – A Modular Tool for Annotating and Analyzing Contributions Tool Demonstrations Jakub Narębski Nicolaus Copernicus University in Toruń, Mikołaj Fejzer Nicolaus Copernicus University in Toruń, Krzysztof Stencel University of Warsaw, Piotr Przymus Nicolaus Copernicus University in Toruń, Poland Link to publication DOI | ||
Cosmos 3C is the third room in the Cosmos 3 wing.
When facing the main Cosmos Hall, access to the Cosmos 3 wing is on the left, close to the stairs. The area is accessed through a large door with the number “3”, which will stay open during the event.