An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment ContextResearch Paper
As use of data driven technologies spreads, software engineers are more often faced with the task of solving a business problem using data-driven methods such as machine learning (ML) algorithms. Deployment of ML within large software systems brings new challenges that are not addressed by standard engineering practices and as a result businesses observe high rate of ML deployment project failures. Data Oriented Architecture (DOA) is an emerging approach that can support data scientists and software developers when addressing such challenges. However, there is a lack of clarity about how DOA systems should be implemented in practice. This paper proposes to consider Flow-Based Programming (FBP) as a paradigm for creating DOA applications. We empirically evaluate FBP in the context of ML deployment on four applications that represent typical data science projects. We use Service Oriented Architecture (SOA) as a baseline for comparison. Evaluation is done with respect to different application domains, ML deployment stages, and code quality metrics. Results reveal that FBP is a suitable paradigm for data collection and data science tasks, and is able to simplify data collection and discovery when compared with SOA. We discuss the advantages of FBP as well as the gaps that need to be addressed to increase FBP adoption as a standard design paradigm for DOA.
Mon 16 MayDisplayed time zone: Eastern Time (US & Canada) change
09:30 - 11:00 | Training & LearningCAIN 2022 at CAIN main room Chair(s): Jan Bosch Chalmers University of Technology | ||
09:30 15mResearch paper | An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment ContextResearch Paper CAIN 2022 Andrei Paleyes Department of Computer Science and Technology, Univesity of Cambridge, Christian Cabrera Department of Computer Science and Technology, Univesity of Cambridge, Neil D. Lawrence Department of Computer Science and Technology, Univesity of Cambridge Pre-print Media Attached | ||
09:45 15mResearch paper | Automatic Checkpointing and Deterministic Training for Deep LearningResearch Paper CAIN 2022 Xiangzhe Xu Purdue University, Hongyu Liu Huawei Galois Lab, China, Guanhong Tao Purdue University, USA, Zhou Xuan Purdue University, Xiangyu Zhang Purdue University | ||
10:00 15mResearch paper | Influence-Driven Data Poisoning in Graph-Based Semi-Supervised ClassifiersResearch Paper CAIN 2022 Adriano Franci University of Luxembourg, Maxime Cordy University of Luxembourg, Luxembourg, Martin Gubri University of Luxembourg, Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg | ||
10:15 15mIndustry talk | Engineering a Platform for Reinforcement Learning WorkloadsIndustry Talk CAIN 2022 | ||
10:30 15mResearch paper | Method Cards for Prescriptive Machine-Learning TransparencyResearch Paper CAIN 2022 David Adkins Meta AI, Bilal Alsallakh Meta AI, Adeel Cheema Meta AI, Narine Kokhlikyan Meta AI, Emily McReynolds Meta AI, Pushkar Mishra Meta AI, Chavez Procope Meta AI, Jeremy Sawruk Meta AI, Erin Wang Meta AI, Polina Zvyagina Meta AI | ||
10:45 15mOther | Discussion on Training & Learning CAIN 2022 |