Explainable Human-Machine Teaming using Model Checking and Interpretable Machine Learning
The human-machine teaming paradigm promotes tight teamwork between humans and autonomous machines that collaborate in the same physical space. This paradigm is increasingly widespread in critical domains, such as healthcare and domestic assistance. These systems are expected to build a certain level of trust by enforcing dependability and exhibiting interpretable behavior. However, trustworthiness is negatively affected by the black-box nature of these systems, which typically make fully autonomous decisions that may be confusing for humans or cause hazards in critical domains. We present the EASE approach, whose goal is to build better trust in human-machine teaming leveraging statistical model checking and model-agnostic interpretable machine learning techniques. We illustrate EASE through an example in the healthcare domain featuring an infinite (dense) space of human-machine uncertain factors, such as diverse physical and physiological characteristics of the agents involved in the teamwork. Our empirical evaluation demonstrates the suitability and cost-effectiveness of EASE in explaining dependability properties in human-machine teaming.
Sun 14 MayDisplayed time zone: Hobart change
11:00 - 12:30 | Synthesis and AIFormaliSE 2023 at Meeting Room 102 Chair(s): Mark Utting The University of Queensland | ||
11:00 30mTalk | Goal Controller Synthesis for Self-Adaptive Systems FormaliSE 2023 | ||
11:30 30mPaper | Verifying Binary Neural Networks on Continuous Input Space using Star Reachability FormaliSE 2023 Mykhailo Ivashchenko University of Nebraska-Lincoln, Sung Woo Choi , Luan Nguyen University of Pennsylvania, Hoang-Dung Tran Vanderbilt University | ||
12:00 30mPaper | Explainable Human-Machine Teaming using Model Checking and Interpretable Machine Learning FormaliSE 2023 Marcello Bersani Politecnico di Milano, Italy, Matteo Camilli Politecnico di Milano, Livia Lestingi DEIB, Politecnico di Milano, Raffaela Mirandola Politecnico di Milano, Matteo Rossi Politecnico di Milano |