The effectiveness of the testing process to reveal software defects not only depends on the characteristics of the test inputs and how thoroughly these exercise the software, but also on the quality of the oracles used to determine whether the software behaves as expected or not. Therefore, assessing the quality of oracles is crucial to improve the overall effectiveness of the testing process. Direct and indirect metrics have been used to assess oracle quality, but they either lack the provision of a comprehensive output that can be used to guide the improvement of the oracles, or are designed for specific types of oracles, thus lacking generality.

In this paper, we introduce \emph{state field coverage}, a novel metric to assess the quality of oracles. Essentially, the state field coverage metric measures the proportion of the objects states, as statically defined in the corresponding classes, that may be accessed by an oracle during test execution. The main intuition of our metric is that oracles with a higher state field coverage are more likely to detect faults in the software under analysis, as they inspect a large portion of the object states to determine whether tests pass or not.

We implement a mechanism to statically compute our state field coverage metric. As a statically computed metric, it can be computed efficiently, and provides direct guidance on how to improve test oracles, by pointing to the state field definitions that are not examined by the test oracles. We also evaluate state field coverage in a series of experiments comprising oracles realized in 273 representation invariants and 249,027 test assertions. Our results show that state field coverage is a well-suited metric for assessing the quality of oracles, since it is highly correlated with the ability of the oracles to detect faults, as measured by mutation score.