On the Cost-Effectiveness of Composite Metamorphic Relations for Testing Deep Learning Systems
Deep Learning (DL) components are increasing their presence in mission and safety-critical systems, such as autonomous vehicles. The verification process of such systems needs to be rigorous, for which automated solutions are paramount. To allow test automation, test oracles are necessary. In the context of DL systems, metamorphic test oracles have found to be effective. However, such oracles require the execution of multiple tests, which makes testing more expensive. Metamorphic relation composition can reduce the cost of metamorphic testing. However, its effectiveness has found mixed answers. This paper reports the preliminary results of our study on measuring the cost-effectiveness of composite metamorphic relations for testing DL systems. To this end, we empirically evaluate the cost-effectiveness of composite metamorphic relations within a DL model for object classification. Our results suggest that composite metamorphic relations reduce the failure revealing capability when compared to their component metamorphic relations.
Mon 9 MayDisplayed time zone: Eastern Time (US & Canada) change
13:00 - 14:00 | |||
13:00 30mTalk | On the Cost-Effectiveness of Composite Metamorphic Relations for Testing Deep Learning Systems MET Aitor Arrieta Mondragon University | ||
13:30 30mTalk | Automated Generation of Metamorphic Relations for Query-Based Systems MET Sergio Segura Universidad de Sevilla, Juan C. Alonso Universidad de Sevilla, Alberto Martin-Lopez Universidad de Sevilla, Amador Durán University of Seville, Javier Troya Universidad de Málaga, Spain, Antonio Ruiz-Cortés University of Seville |