Every program has a corresponding input domain. Thus, every faulty program has an input domain with failure patterns which consist of failure causing inputs. We define failure-based testing as testing methods that make use of the information about various aspects of failure patterns, such as their shapes, sizes, orientations, sizes and numbers. With such information, we are able to design new testing methods and to better understand the inter-relations between some apparently unrelated results. The information of failure patterns not only helps to test programs but also can help to do fault localisation and program repair.

Adaptive random testing is a failure-based testing method. However, adaptive random testing only makes use of one aspect of failure patterns, namely the contiguity of failure-causing inputs. White and Cohen’s domain strategy (1980) which was proposed as a fault-based testing method, can be regarded as a failure-based testing method. Apart from sharing some interesting and important software testing results, we would like to address the potential and future of failure-based testing, in particular, in the era of AI. We believe failure-based testing has lots of research opportunities.