A Framework for Including Uncertainty in Robustness Evaluation of Bayesian Neural Network Classifiers
Neural networks (NNs) play a crucial role in safety-critical fields, requiring robustness assurance. Bayesian Neural Networks (BNNs) address data uncertainty, providing probabilistic outputs. However, the literature on BNN robustness assessment is still limited, mainly focusing on adversarial examples, which are often impractical in real-world applications. This paper introduces a fresh perspective on BNN classifier robustness, considering natural input variations while accounting for prediction uncertainties. Our approach excludes predictions labeled as “unknown”, enabling practitioners to define alteration probabilities, penalize errors beyond a specified threshold, and tolerate varying error levels below it. We present a systematic approach for evaluating the robustness of BNNs, introducing new evaluation metrics that account for prediction uncertainty. We conduct a comparative study using two NNs – standard MLP and Bayesian MLP – on the MNIST dataset. Our results show that by leveraging estimated uncertainty, it is possible to enhance the system’s robustness
Sat 20 AprDisplayed time zone: Lisbon change
11:00 - 12:30 | Research TalksDeepTest at Eugénio de Andrade Chair(s): Matteo Biagiola Università della Svizzera italiana | ||
11:00 30mPaper | Data vs. Model Machine Learning Fairness Testing: An Empirical Study DeepTest Arumoy Shome Delft University of Technology, Luís Cruz Delft University of Technology, Arie van Deursen Delft University of Technology Pre-print | ||
11:30 30mPaper | Guiding the Search Towards Failure-Inducing Test Inputs Using Support Vector Machines DeepTest Lev Sorokin fortiss GmbH | Technische Universität München, Niklas Kerscher Technische Universität München | Ludwig-Maximilians-Universität München Pre-print | ||
12:00 30mPaper | A Framework for Including Uncertainty in Robustness Evaluation of Bayesian Neural Network Classifiers DeepTest Wasim Essbai Technische Universität Wien, Andrea Bombarda University of Bergamo, Silvia Bonfanti University of Bergamo, Angelo Gargantini University of Bergamo Pre-print |