There are two key challenges in automatically testing web APIs: (a) determine where to send API requests and (b) identify how to make a valid payload for a given request. Both challenges are sometimes addressed by the presence of a machine-parseable API specification (such as an OpenAPI specification). However, most web applications lack such a specification—making automatic testing hard.

We tackle both challenges by introducing Trailblazer, a practical end-to-end workflow for testing web APIs—regardless of whether a specification is available. Trailblazer operates by (1) capturing API requests initiated by the client side of a web application during normal interactions, (2) identifying endpoints and inferring request payload structure using the collected data, and then (3) generating new test payloads. To the best of our knowledge, Trailblazer is the first of its kind to combine generation-based and mutation-based fuzzing in web API testing.

We evaluated Trailblazer against popular open-source content management systems, found that the code coverage it achieved, was comparable to the coverage obtained using official OpenAPI specification. Trailblazer uncovered seven unique new bugs across the tested systems, with two already fixed and four confirmed.