Write a Blog >>
ISSTA 2020
Sat 18 - Wed 22 July 2020
Tue 21 Jul 2020 15:10 - 15:30 at Zoom - REGRESSION TESTING Chair(s): Alex Orso

Cloud services are programmatically accessed through REST APIs. Since REST APIs are constantly evolving, an important problem is how to prevent breaking changes of APIs, while supporting several different versions. To find such breaking changes in an automated way, we introduce differential regression testing for REST APIs. Our approach is based on two main observations. First, breaking changes in REST APIs involve two software components, namely the client and the service. As such, we observe that there are also two types of regressions: regressions in the API specification, i.e., in the contract between the client and the service, and regressions in the service itself, i.e., previously working requests are “broken” in later versions of the service. Finding both kinds of regressions involves testing along two dimensions: when the service changes and when the specification changes. Second, to detect such bugs automatically, we employ differential testing. That is, we compare the behavior of different versions on the same inputs against each other, and find regressions in the observed differences. For generating inputs (sequences of HTTP requests) to services, we use RESTler, a stateful fuzzer for REST APIs. Comparing the outputs (HTTP responses) of a cloud service involves several challenges, like abstracting over minor differences, handling out-of-order requests, and non-determinism. Differential regression testing across 17 different versions of the widely used Azure networking APIs deployed between 2016 and 2019 detected 14 regressions in total, 5 of those in the official API specifications and 9 regressions in the services themselves.

Tue 21 Jul

Displayed time zone: Tijuana, Baja California change

14:50 - 15:50
REGRESSION TESTINGTechnical Papers at Zoom
Chair(s): Alex Orso Georgia Institute of Technology

Public Live Stream/Recording. Registered participants should join via the Zoom link distributed in Slack.

14:50
20m
Talk
Dependent-Test-Aware Regression Testing Techniques
Technical Papers
Wing Lam University of Illinois at Urbana-Champaign, August Shi The University of Texas at Austin, Reed Oei , Sai Zhang Google Cloud, Michael D. Ernst University of Washington, USA, Tao Xie Peking University
DOI Media Attached
15:10
20m
Talk
Differential Regression Testing for REST APIs
Technical Papers
Patrice Godefroid Microsoft Research, Daniel Lehmann University of Stuttgart, Marina Polishchuk Microsoft
DOI Media Attached
15:30
20m
Talk
Empirically Revisiting and Enhancing IR-Based Test-Case Prioritization
Technical Papers
Qianyang Peng , August Shi The University of Texas at Austin, Lingming Zhang The University of Texas at Dallas
DOI