ICSE 2024
Fri 12 - Sun 21 April 2024 Lisbon, Portugal
Sat 20 Apr 2024 11:00 - 11:30 at Eugénio de Andrade - Research Talks Chair(s): Matteo Biagiola

Although several fairness definitions and bias mitigation techniques exist in the literature, all existing solutions evaluate fairness of Machine Learning (ML) systems after the training stage. In this paper, we take the first steps towards evaluating a more holistic approach by testing for fairness both before and after model training. We evaluate the effectiveness of the proposed approach and position it within the ML development lifecycle, using an empirical analysis of the relationship between model dependent and independent fairness metrics. The study uses 2 fairness metrics, 4 ML algorithms, 5 real-world datasets and 1600 fairness evaluation cycles. We find a linear relationship between data and model fairness metrics when the distribution and the size of the training data changes. Our results indicate that testing for fairness prior to training can be a “cheap” and effective means of catching a biased data collection process early; detecting data drifts in production systems and minimising execution of full training cycles thus reducing development time and costs.

Sat 20 Apr

Displayed time zone: Lisbon change

11:00 - 12:30
Research TalksDeepTest at Eugénio de Andrade
Chair(s): Matteo Biagiola Università della Svizzera italiana
11:00
30m
Paper
Data vs. Model Machine Learning Fairness Testing: An Empirical Study
DeepTest
Arumoy Shome Delft University of Technology, Luís Cruz Delft University of Technology, Arie van Deursen Delft University of Technology
Pre-print
11:30
30m
Paper
Guiding the Search Towards Failure-Inducing Test Inputs Using Support Vector Machines
DeepTest
Lev Sorokin fortiss GmbH | Technische Universität München, Niklas Kerscher Technische Universität München | Ludwig-Maximilians-Universität München
Pre-print
12:00
30m
Paper
A Framework for Including Uncertainty in Robustness Evaluation of Bayesian Neural Network Classifiers
DeepTest
Wasim Essbai Technische Universität Wien, Andrea Bombarda University of Bergamo, Silvia Bonfanti University of Bergamo, Angelo Gargantini University of Bergamo
Hide past events