Systematic API Testing Through Model Checking and Executable Contracts

Automated black-box testing of APIs typically relies on interface specifications that define available operations and data schemas but offer limited or no behavioural semantics. This semantic gap amplifies the test-oracle problem - determining whether observed responses and resulting states are correct - and limits the generation of effective, stateful call sequences.
We introduce IcePick, a framework that achieves systematic state-space coverage for API testing by leveraging model checking. IcePick uses TLA+ to formally model API state evolution, employs the TLC model checker to exhaustively explore reachable states, and generates test sequences that provably cover the behavioural model. To mitigate state-space explosion and improve sequence extraction, we introduce a coverage-guided breadth-first traversal of the TLC state-space graph.
To address oracle limitations beyond HTTP status codes, we propose Glacier, a first-order logic contract language that enriches API specifications with executable semantic contracts, enabling automated behavioural verification during test execution.
We evaluate IcePick on EvoMaster Benchmark systems, demonstrating that model-checking-guided exploration achieves complete state coverage and uncovers bugs in multi-operation interactions. We also analyse scalability to characterise practical limits and applicability requirements.
Overall, IcePick provides reproducible test suites with strong coverage guarantees for critical API-based systems.
Wed 20 MayDisplayed time zone: Seoul change
10:30 - 12:00 | Specification Inference & Model CheckingJournal-First Papers / Research Papers at Room 103 Chair(s): Eunkyoung Jee KAIST, South Korea | ||
10:30 25mTalk | GRANDSLAM: Linearly Scalable Model Synthesis Research Papers Alexander Boll University of Bern | ||
10:55 25mTalk | Improving Dynamic Specification Inference with LLM-Generated Counterexamples Research Papers AgustÃn Balestra University of Rio Cuarto, Argentina, Agustin Nolasco University of Rio Cuarto, Facundo Molina Complutense University of Madrid, Diego Garbervetsky Departamento de Computación, FCEyN, UBA, Renzo Degiovanni Luxembourg Institute of Science and Technology, Nazareno Aguirre University of Rio Cuarto/CONICET, Argentina, and Guangdong Technion-Israel Institute of Technology, China | ||
11:15 25mTalk | Systematic API Testing Through Model Checking and Executable Contracts Research Papers Pre-print | ||
11:40 15mTalk | Simulation-based Safety Assessment of Vehicle Characteristics Variations in Autonomous Driving Systems Journal-First Papers Qi Pan Nanjing University of Aeronautics and Astronautics, Tiexin Wang Nanjing University of Aeronautics and Astronautics, Jianwei Ma Nanjing University of Aeronautics and Astronautics, Paolo Arcaini National Institute of Informatics, Tao Yue Beihang University Link to publication DOI | ||