Write a Blog >>
Thu 13 Jul 2017 16:30 - 17:00 at ESB 2003 - Testing Chair(s): Alex Groce

The oracle problem remains one of the key challenges in software testing, for which little automated support has been developed so far. In my thesis work we introduce a technique for assessing and improving test oracles by reducing the incidence of both false positives and false negatives. Our technique combines test case generation to reveal false positives and mutation testing to reveal false negatives. The experimental results on five real-world subjects show that the fault detection rate of the oracles after improvement increases, on average, by 48.6% (86% over the implicit oracle). Three actual, exposed faults in the studied systems were subsequently confirmed and fixed by the developers. However, our technique contains a human in the loop, which was represented only by the author during the initial experiments. Our next goal is to conduct further experiments where the human in the loop will be represented by real developers. Our second future goal is to address the oracle placement problem. When testing software, developers can place oracles externally or internally to a method. Given a faulty execution state, i.e., one that differs from the expected one, an oracle might be unable to expose the fault if it is placed at a program point with no access to the incorrect program state or where the program state is no longer corrupted. In such a case, the oracle is subject to failed error propagation. Internal oracles are in principle less subject to failed error propagation than external oracles. However, they are also more difficult to define manually. Hence, a key research question is whether a more intrusive oracle placement is justified by its higher fault detection capability.

Thu 13 Jul

Displayed time zone: Tijuana, Baja California change

15:30 - 17:00
TestingDoctoral Symposium at ESB 2003
Chair(s): Alex Groce Northern Arizona University
15:30
30m
Talk
Data Flow Oriented UI Testing
Doctoral Symposium
Nataniel Borges Jr. Saarland University
16:00
30m
Talk
Dynamic Tainting for Automatic Test Case Generation
Doctoral Symposium
A: Björn Mathis Saarland University
16:30
30m
Talk
Oracle Problem in Software Testing
Doctoral Symposium
A: Gunel Jahangirova Fondazione Bruno Kessler