Quantifying Faultiness: What Does It Mean to Have N Faults
It is common to see software testing experiments where a benchmark program is seeded with N benchmark mutations, then declared to have N faults. We argue that in the absence of a formal definition of what is a fault, let alone what are N faults, it is difficult to assign a meaning to such claims. We further argue that in order to assign a meaning to a claim such as “Program P has N faults”, we need to define what is a fault, and what is a unitary fault; also we find that such a claim depends on a number of parameters that must be made explicit for the claim to be meaningful. Finally, we propose a set of metrics that reflect program faultiness, and contrast them to current practice, using the tcas component of the Siemens benchmark as an illustrative example.
Fri 21 MayDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
15:30 - 17:00 | |||
15:30 30mTalk | Runtime Verification Under Access Restrictions FormaliSE 2021 Rania Taleb PhD student, Raphael Khoury Université du Québec à Chicoutimi, Canada, Sylvain Hallé Université du Québec à Chicoutimi Pre-print Media Attached | ||
16:00 30mTalk | Quantifying Faultiness: What Does It Mean to Have N Faults FormaliSE 2021 Media Attached | ||
16:30 30mTalk | Checking temporal patterns of API usage without code execution FormaliSE 2021 Erick Raelijohn University of Montreal, Michalis Famelis Université de Montréal, Houari Sahraoui Université de Montréal Media Attached |
Go directly to this room on Clowdr