TestifAI: Probabilistic Context-Aware Testing For Safe Deep Learning Models
Deep neural networks (DNNs), are integral to AI systems deployed in safety-critical domains like autonomous vehicles, healthcare, and cybersecurity. However, assessing robustness in real-world scenarios remains a significant challenge. Traditional metrics like neuron coverage often fail to capture the sensitivity of DNN models towards input perturbations and increase the risk of unexpected failures. This paper introduces a probabilistic framework, TestifAI, to comprehensively evaluate and enhance DNN robustness across diverse and context-sensitive scenarios. TestifAI includes four key stages: (1) specification where, for a given DNN model and dataset, users specify test criteria in the form of robustness properties (e.g., sensitivity to image rotation), their configuration (e.g., range of rotation angles), and statistical dependencies between properties and configurations to simulate real-world scenarios; (2) mapping constructs a probabilistic coverage graph that captures the aforementioned dependencies, updating probabilities as testing progresses; (3) test-case generation systematically produces targeted unit tests aligned with specified robustness properties given their configuration; and, finally, (4) execution where, given tests, the framework calculates both local (property-specific) and global (system-wide, based on dependencies) coverage metrics. Experimental results demonstrate that the TestifAI framework offers a rigorous and adaptable approach to DNN testing, suitable for complex, mission-critical, and safety-sensitive environments.
Tue 29 AprDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 17:00 | Session 4: Testing (talks and panel)Doctoral Symposium at 212 Chair(s): Tayana Conte Universidade Federal do Amazonas | ||
16:00 6mTalk | TestifAI: Probabilistic Context-Aware Testing For Safe Deep Learning Models Doctoral Symposium AroojArif Northeastern University London | ||
16:06 6mTalk | Foundation Models for Automatic Issue Labeling Doctoral Symposium Giuseppe Colavito University of Bari | ||
16:12 6mTalk | Automatically Generating Single-Responsibility Unit Tests Doctoral Symposium Geraldine Galindo-Gutierrez Centro de Investigación en Ciencias Exactas e Ingenierías, Universidad Católica Boliviana | ||
16:18 6mTalk | Automatic Test Case Generation for Smart Human-Centric Ecosystems Doctoral Symposium Alind Xhyra Universitá della Svizzera Italiana (USI) Lugano, Constructor Institute of Technology (CIT) Schaffhausen | ||
16:24 6mTalk | A Framework for On the Fly Input Refinement for Deep Learning Models Doctoral Symposium Ravishka Shemal Rathnasuriya University of Texas at Dallas | ||
16:30 30mPanel | Panel: Testing Doctoral Symposium Shaukat Ali Simula Research Laboratory and Oslo Metropolitan University, Xavier Devroey University of Namur, Annibale Panichella Delft University of Technology, Ahmed Arif University of California, Merced, Giuseppe Colavito University of Bari, Geraldine Galindo-Gutierrez Centro de Investigación en Ciencias Exactas e Ingenierías, Universidad Católica Boliviana, Ravishka Shemal Rathnasuriya University of Texas at Dallas, Alind Xhyra Universitá della Svizzera Italiana (USI) Lugano, Constructor Institute of Technology (CIT) Schaffhausen |