ML-On-Rails: Safeguarding Machine Learning Models in Software Systems – A Case Study
Machine learning (ML), especially with the emergence of large language models (LLMs), has significantly transformed various industries. However, the transition from ML model prototyping to production use within software systems presents several challenges. These challenges primarily revolve around ensuring safety, security, and transparency, subsequently influencing the overall robustness and trustworthiness of ML models. In this paper, we introduce ML-On-Rails, a protocol designed to safeguard ML models, establish a well-defined endpoint interface for different ML tasks, and clear communication between ML providers and ML consumers (software engineers). ML-On-Rails enhances the robustness of ML models via incorporating detection capabilities to identify unique challenges specific to production ML. We evaluated the ML-On-Rails protocol through a real-world case study of the MoveReminder application. Through this evaluation, we emphasize the importance of safeguarding ML models in production.
Mon 15 AprDisplayed time zone: Lisbon change
14:00 - 15:30 | |||
14:00 15mTalk | A Combinatorial Testing Approach to Hyperparameter OptimizationDistinguished paper Award Candidate Research and Experience Papers Krishna Khadka The University of Texas at Arlington, Jaganmohan Chandrasekaran Virginia Tech, Jeff Yu Lei University of Texas at Arlington, Raghu Kacker National Institute of Standards and Technology, D. Richard Kuhn National Institute of Standards and Technology | ||
14:15 15mTalk | Mutation-based Consistency Testing for Evaluating the Code Understanding Capability of LLMs Research and Experience Papers | ||
14:30 10mTalk | LLMs for Test Input Generation for Semantic Applications Research and Experience Papers Zafaryab Rasool Applied Artificial Intelligence Institute, Deakin University, Scott Barnett Applied Artificial Intelligence Institute, Deakin University, David Willie Applied Artificial Intelligence Institute, Deakin University, Stefanus Kurniawan Deakin University, Sherwin Balugo Applied Artificial Intelligence Institute, Deakin University, Srikanth Thudumu Deakin University, Mohamed Abdelrazek Deakin University, Australia | ||
14:40 10mTalk | (Why) Is My Prompt Getting Worse? Rethinking Regression Testing for Evolving LLM APIs Research and Experience Papers MA Wanqin The Hong Kong University of Science and Technology, Chenyang Yang Carnegie Mellon University, Christian Kästner Carnegie Mellon University | ||
14:50 10mTalk | Welcome Your New AI Teammate: On Safety Analysis by Leashing Large Language Models Research and Experience Papers Ali Nouri Volvo cars & Chalmers University of Technology, Beatriz Cabrero-Daniel University of Gothenburg, Fredrik Torner Volvo cars, Hakan Sivencrona Zenseact AB, Christian Berger Chalmers University of Technology, Sweden | ||
15:00 10mTalk | ML-On-Rails: Safeguarding Machine Learning Models in Software Systems – A Case Study Research and Experience Papers Hala Abdelkader Applied Artificial Intelligence Institute, Deakin University, Mohamed Abdelrazek Deakin University, Australia, Scott Barnett Applied Artificial Intelligence Institute, Deakin University, Jean-Guy Schneider Monash University, Priya Rani RMIT University, Rajesh Vasa Deakin University, Australia | ||
15:10 20mLive Q&A | Test - Q&A Session Research and Experience Papers |