CAIN 2024
Sun 14 - Mon 15 April 2024 Lisbon, Portugal
co-located with ICSE 2024

Datasets and models are two key artifacts in machine learning (ML) applications. Although there exist tools to support dataset and model developers in managing ML artifacts, little is known about how these datasets and models are integrated into ML applications. In this paper, we study how datasets and models in ML applications are managed. In particular, we focus on how these artifacts are stored and versioned alongside the applications. After analyzing 93 repositories, we identified the most common storage location to store datasets and models is the file system, which causes availability issues. Notably, large data and model files, exceeding approximately 60 MB, are stored exclusively in remote storage. Most of the datasets and models lack proper integration with the version control system, posing potential traceability and reproducibility issues. Additionally, although datasets and models are likely to evolve during the application development, they are rarely updated in application repositories

Sun 14 Apr

Displayed time zone: Lisbon change

14:00 - 15:30
Data Engineering and Management for AI-Enabled SystemsResearch and Experience Papers / Industry Talks at Pequeno Auditório
Chair(s): Marc Zeller Siemens AG
14:00
15m
Talk
What About the Data? A Mapping Study on Data Engineering for AI Systems
Research and Experience Papers
Petra Heck Fontys University of Applied Sciences
Pre-print
14:15
15m
Talk
Unmasking Data Secrets: An Empirical Investigation into Data Smells and Their Impact on Data Quality
Research and Experience Papers
Gilberto Recupito University of Salerno, Raimondo Rapacciuolo University of Salerno, Dario Di Nucci University of Salerno, Fabio Palomba University of Salerno
14:30
15m
Talk
An Exploratory Study of Dataset and Model Management in Open Source Machine Learning ApplicationsDistinguished paper Award Candidate
Research and Experience Papers
Tajkia Rahman Toma University of Alberta, Cor-Paul Bezemer University of Alberta
14:45
10m
Talk
DVC in Open Source AI-development: The Action and the Reaction
Research and Experience Papers
Lorena Barreto Simedo Pacheco Concordia University, Musfiqur Rahman Concordia University, Fazle Rabbi Concordia University, Pouya Fathollahzadeh Queen’s University, Ahmad Abdellatif University of Calgary, Emad Shihab Concordia University, Tse-Hsun (Peter) Chen Concordia University, Jinqiu Yang Concordia University, Ying Zou Queen's University, Kingston, Ontario
14:55
10m
Industry talk
Structuring the world of unstructured text data – Balancing business requirements, training data availability, and model performance.
Industry Talks
15:05
10m
Industry talk
Invited: Artificial Intelligence Projects, a quest between meaningful use cases, data, and unfulfilled desires.
Industry Talks
A: Andreas Jedlitschka Fraunhofer IESE
15:15
15m
Live Q&A
Data : Q&A Session
Research and Experience Papers