ICST 2025
Mon 31 March - Fri 4 April 2025 Naples, Italy
Tue 1 Apr 2025 11:30 - 12:00 at Room C - Technical Program

As the usage of Artificial Intelligence (AI) on resource-intensive and safety-critical tasks increases, a variety of Machine Learning (ML) compilers have been developed, enabling compatibility of Deep Neural Networks (DNNs) with a variety of hardware acceleration devices. However, given that DNNs are widely utilized for challenging and demanding tasks, the behavior of these compilers must be verified. To this direction, we propose a tool that utilizes elements of both differential and mutation testing in order to examine the robustness of image recognition models when deployed on hardware accelerators with different capabilities, in the presence of faults in their target device code - introduced either by developers, or problems in their compilation process. We focus on the image recognition domain by applying mutation testing to 7 well-established DNN models, introducing 21 mutations of 6 different categories. We deployed our mutants on 4 different hardware acceleration devices of varying capabilities and observed that DNN models presented discrepancies of up to 90.3% in mutants related to conditional operators across devices. We also observed that mutations related to layer modification, arithmetic types and input affected severely the overall model performance (up to 99.8%) or led to model crashes, in a consistent manner across devices.

Tue 1 Apr

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

11:00 - 12:30
Technical ProgramMutation at Room C
11:00
30m
Paper
Equivalent Mutants: Deductive Verification to the Rescue
Mutation
Serge Demeyer University of Antwerp and Flanders Make vzw, Reiner Hähnle Technical University of Darmstadt
11:30
30m
Paper
Exploring Robustness of Image Recognition Models on Hardware Accelerators
Mutation
Nikolaos Louloudakis University of Edinburgh, Perry Gibson University of Glasgow, José Cano University of Glasgow, Ajitha Rajan University of Edinburgh
12:00
30m
Paper
Semantic-Preserving Transformations as Mutation Operators: A Study on Their Effectiveness in Defect Detection
Mutation
Max Hort Simula Research Laboratory, Linas Vidziunas , Leon Moonen Simula Research Laboratory
:
:
:
: