When improving their code, developers often turn to interactive debuggers. The correctness of these tools is crucial, because bugs in the debugger itself may mislead a developer, e.g., to believe that executed code is never reached or that a variable has another value than in the actual execution. Yet, debuggers are difficult to test because their input consists of both source code and a sequence of debugging actions, such as setting breakpoints or stepping through code. This paper presents the first metamorphic testing approach for debuggers. The key idea is to transform both the debugged code and the debugging actions in such a way that the behavior of the original and the transformed inputs should differ only in specific ways. For example, adding a breakpoint should not change the flow of control of the debugged program. To support the interactive nature of debuggers, we introduce interactive metamorphic testing. It differs from traditional metamorphic testing by determining the input transformation and the expected behavioral change it causes while the program under test is running. Our evaluation applies the approach to the widely used debugger in the Chromium browser, where it finds previously eight unknown bugs with a true positive rate of 51%. All bugs have been confirmed by the developers, and two bugs have even been marked as release-blocking.
Fri 19 JulDisplayed time zone: Beijing, Chongqing, Hong Kong, Urumqi change
11:00 - 12:30 | Static Analysis and DebuggingTechnical Papers at Grand Ballroom Chair(s): Arie van Deursen Delft University of Technology | ||
11:00 22mTalk | Differentially Testing Soundness and Precision of Program Analyzers Technical Papers Christian Klinger University of Texas, Austin, Maria Christakis MPI-SWS, Valentin Wüstholz ConsenSys Diligence Pre-print | ||
11:22 22mTalk | Judge: Identifying, Understanding, and Evaluating Sources of Unsoundness in Call Graphs Technical Papers Michael Reif TU Darmstadt, Germany, Florian Kübler TU Darmstadt, Germany, Michael Eichberg TU Darmstadt, Germany, Dominik Helm TU Darmstadt, Germany, Mira Mezini TU Darmstadt, Germany Pre-print File Attached | ||
11:45 22mTalk | Adlib: Analyzer for Mobile Ad Platform Libraries Technical Papers DOI Pre-print | ||
12:07 22mTalk | Interactive Metamorphic Testing of Debuggers Technical Papers Sandro Tolksdorf TU Darmstadt, Daniel Lehmann TU Darmstadt, Michael Pradel TU Darmstadt and Facebook Link to publication DOI Pre-print |