Trustworthy artificial intelligence (Trusted AI) is essential when autonomous, safety-critical systems use learning-enabled components (LECs) in uncertain environments. When reliant on deep learning, these learning-enabled self-adaptive systems (LESAS) must address the reliability, interpretability, and robustness (collectively, the assurance) of learning models. Three types of uncertainty most significantly affect assurance. First, uncertainty about the physical environment can cause suboptimal, and sometimes catastrophic, results as the system struggles to adapt to unanticipated or poorly-understood environmental conditions. For example, when lane markings are occluded (either on the camera and/or the physical lanes), lane management functionality can be critically compromised. Second, uncertainty in the cyber environment can create unexpected and adverse consequences, including not only performance impacts (network load, real-time responses, etc.) but also potential threats or overt (cybersecurity) attacks. Third, uncertainty can exist with the components themselves and affect how they interact upon reconfiguration. Left unchecked, it may cause unexpected and unwanted feature interactions. While learning-enabled technologies have made great strides in addressing uncertainty, challenges remain in addressing the assurance of such systems when encountering uncertainty not addressed in training data. Furthermore, we need to consider LESASs as first-class software-based systems that should be rigorously developed, verified, and maintained — i.e., software engineered. In addition to developing specific strategies to address these concerns, appropriate software architectures are needed to coordinate LECs and ensure they deliver acceptable behavior even under uncertain conditions. To this end, this presentation overviews a number of our multi-disciplinary research projects, involving industrial collaborators, which collectively support a software engineering, model-based approach to address Trusted AI and provide assurance for learning-enabled self-adaptive systems (i.e., SE4LESAS). In addition to sharing lessons learned from more than two decades of research addressing assurance for (learning-enabled) self-adaptive systems operating under a range of uncertainty, near-term and longer-term research challenges for SE4LESAS will be overviewed.

Betty H.C. Cheng is a Professor in the Department of Computer Science and Engineering at Michigan State University. Her research interests include trusted AI, automated software engineering, self-adaptive systems, requirements engineering, model-driven engineering, automotive cyber security, search-based software engineering. These research areas are used to support the development of high-assurance self-adaptive systems that must continuously deliver acceptable behavior, even in the face of environmental and system uncertainty. Example applications include intelligent transportation and vehicle systems. She collaborates extensively with industrial partners in her research projects in order to facilitate technology exchange between academia and industry. Previously, she was awarded a NASA/JPL Faculty Fellowship to investigate the use of new software engineering techniques for a portion of the NASA space shuttle software. Recent work focuses on assurance of learning-enabled systems, cyber security for automotive systems, and feature interaction detection and mitigation for autonomic systems, all in the context of operating under uncertainty while maintaining assurance objectives. Her research has been funded by several federal funding agencies, including NSF, ONR, DARPA, NASA, AFRL, ARO, and numerous industrial organizations. She is an Associate Editor-in-Chief for IEEE Transactions on Software Engineering, having previously served two terms on the editorial board. She also serves on the editorial boards for Requirements Engineering Journal and Software and Systems Modeling. She was the Technical Program Co-Chair for IEEE International Conference on Software Engineering (ICSE-2013), the premier and flagship conference for software engineering.

She received her BS from Northwestern University and her MS and PhD degrees from the University of Illinois-Urbana Champaign, all in computer science. She may be reached at the Department of Computer Science and Engineering, Michigan State University, 3115 Engineering Building, 428 S. Shaw Lane, East Lansing, MI 48824; chengb@msu.edu; https://www.cse.msu.edu/~chengb.