Keynotes page

Developing software systems that contain machine learning (ML) components (ML-enabled systems, or ML systems for short) requires an end-to-end perspective that considers the unique life cycle of these components — from data acquisition, to model training, to model deployment and evolution. However, a problem is that ML system development typically split into three roles, with three different and often separate workflows: data scientists build models; software engineers integrate models into ML systems; and then operations staff deploy, operate, and monitor the ML systems. Because these roles operate separately and often speak different languages, there are opportunities for mismatch between the assumptions made by each role with respect to the elements of the ML-enabled system, and the actual guarantees provided by each element, which leads to system failure. While simply better collaboration between teams is a valid solution to this problem, using software architecture design as the set of activities and artifacts that promote and record collaborative decision making is a much stronger and sustainable solution.

In this talk I will first share the outcomes of two practitioner studies that highlight the problems related to treating ML systems development as a model-centric instead of a system-centric activity. I will then present a set of software architecture practices that can lead to successful ML systems. I will close the talk with some thoughts on remaining gaps in both research and practice that I hope will inspire current and future software engineering and software architecture research.

Grace Lewis is principal researcher and lead of the Tactical and AI-enabled Systems (TAS) initiative at the Software Engineering Institute at Carnegie Mellon University. Grace is the principal investigator for the “Characterizing and Detecting Mismatch in ML-Enabled Systems” and “Predicting Inference Degradation in Production ML Systems” research projects. Her current areas of expertise and interest include software engineering for AI/ML systems, software architecture (in particular the development of software architecture practices for systems that integrate emerging technologies), edge computing, and software engineering in society. She has a B.Sc. in Software Systems Engineering and a Specialization in Administration from Icesi University in Colombia; a Master in Software Engineering from Carnegie Mellon University; and a Ph.D. in Computer Science from Vrije Universiteit Amsterdam. Grace serves as VP for Technical Activities and Conferences (T&C) and member of the Board of Governors for the IEEE Computer Society, Diversity and Inclusion (D&I) chair for the IEEE CS Technical Council on Software Engineering (TCSE), Alternate Representative for IEEE-CS on the ABET CSAB Board of Directors, and an ABET evaluator for Computer Science undergraduate programs.