Method Cards for Prescriptive Machine-Learning TransparencyResearch Paper
Specialized documentation techniques have been developed to communicate key facts about machine-learning (ML) systems and the datasets and models they rely on. Techniques such as Datasheets, FactSheets, and Model Cards have taken a mainly descriptive approach, providing various details about the system components. While the above information is essential for product developers and external experts to assess whether the ML system meets their requirements, other stakeholders might find it less actionable. In particular, ML engineers need guidance on how to mitigate potential shortcomings in order to fix bugs or improve the system’s performance. We propose a documentation artifact that aims to provide such guidance in a prescriptive way. Our proposal, called Method Cards, aims to increase the transparency and reproducibility of ML systems by allowing stakeholders to reproduce the models, understand the rationale behind their designs, and introduce adaptations in an informed way. We showcase our proposal with an example in small object detection, and demonstrate how Method Cards can communicate key considerations that help increase the transparency and reproducibility of the detection model. We further highlight avenues for improving the user experience of ML engineers based on Method Cards.
Mon 16 MayDisplayed time zone: Eastern Time (US & Canada) change
09:30 - 11:00 | Training & LearningCAIN 2022 at CAIN main room Chair(s): Jan Bosch Chalmers University of Technology | ||
09:30 15mResearch paper | An Empirical Evaluation of Flow Based Programming in the Machine Learning Deployment ContextResearch Paper CAIN 2022 Andrei Paleyes Department of Computer Science and Technology, Univesity of Cambridge, Christian Cabrera Department of Computer Science and Technology, Univesity of Cambridge, Neil D. Lawrence Department of Computer Science and Technology, Univesity of Cambridge Pre-print Media Attached | ||
09:45 15mResearch paper | Automatic Checkpointing and Deterministic Training for Deep LearningResearch Paper CAIN 2022 Xiangzhe Xu Purdue University, Hongyu Liu Huawei Galois Lab, China, Guanhong Tao Purdue University, USA, Zhou Xuan Purdue University, Xiangyu Zhang Purdue University | ||
10:00 15mResearch paper | Influence-Driven Data Poisoning in Graph-Based Semi-Supervised ClassifiersResearch Paper CAIN 2022 Adriano Franci University of Luxembourg, Maxime Cordy University of Luxembourg, Luxembourg, Martin Gubri University of Luxembourg, Luxembourg, Mike Papadakis University of Luxembourg, Luxembourg, Yves Le Traon University of Luxembourg, Luxembourg | ||
10:15 15mIndustry talk | Engineering a Platform for Reinforcement Learning WorkloadsIndustry Talk CAIN 2022 | ||
10:30 15mResearch paper | Method Cards for Prescriptive Machine-Learning TransparencyResearch Paper CAIN 2022 David Adkins Meta AI, Bilal Alsallakh Meta AI, Adeel Cheema Meta AI, Narine Kokhlikyan Meta AI, Emily McReynolds Meta AI, Pushkar Mishra Meta AI, Chavez Procope Meta AI, Jeremy Sawruk Meta AI, Erin Wang Meta AI, Polina Zvyagina Meta AI | ||
10:45 15mOther | Discussion on Training & Learning CAIN 2022 |