Fairness Evaluation in Deepfake Detection Models using Metamorphic Testing
Fairness of deepfake detectors in the presence of anomalies are not well investigated, especially if those anomalies are more prominent in either male or female subjects. The primary motivation for this work is to evaluate how deepfake detection model behaves under such anomalies. However, due to the black-box nature of deep learning (DL) and artificial intelligence (AI) systems, it is hard to predict the performance of a model when the input data is modified. Crucially, if this defect is not addressed properly, it will adversely affect the fairness of the model and result in discrimination of certain sub-population unintentionally. Therefore, the objective of this work is to adopt metamorphic testing to examine the reliability of the selected deepfake detection model, and how the transformation of input variation places influence on the output. We have chosen MesoInception-4, a state-of-the-art deepfake detection model, as the target model and makeup as the anomalies.
Mon 9 MayDisplayed time zone: Eastern Time (US & Canada) change
08:00 - 09:30
|In-Place Metamorphic Exploration and Testing|
|Fairness Evaluation in Deepfake Detection Models using Metamorphic Testing|
|SR-MT:A Metamorphic Method to Test Robustness of Speech Recognition Software|