CAIN 2025
Sun 27 - Mon 28 April 2025 Ottawa, Ontario, Canada
co-located with ICSE 2025
Mon 28 Apr 2025 14:55 - 15:10 at 208 - Quality Assurance for AI systems Chair(s): Eduardo Santana de Almeida

[Context] Technical debt (TD) in machine learning (ML) systems, much like its counterpart in software engineering (SE), holds the potential to lead to future rework, posing risks to productivity, quality, and team morale. Despite growing attention to TD in SE, the understanding of ML-specific code-related TD remains underexplored. [Objective] This paper aims to identify and discuss the relevance of code-related issues that lead to TD in ML code throughout the ML life cycle. [Method] The study first compiled a list of 34 potential issues contributing to TD in ML code by examining the phases of the ML life cycle, their typical associated activities, and problem types. This list was refined through two focus group sessions involving nine experienced ML professionals, where each issue was assessed based on its occurrence contributing to TD in ML code and its relevance. [Results] The list of issues contributing to TD in the source code of ML systems was refined from 34 to 30, with 24 of these issues considered highly relevant. The data pre-processing phase was the most critical, with 14 issues considered highly relevant. Shortcuts in code related to typical pre-processing tasks (e.g., handling missing values, outliers, inconsistencies, scaling, rebalancing, and feature selection) often result in “patch fixes” rather than sustainable solutions, leading to the accumulation of TD and increasing maintenance costs. Relevant issues were also found in the data collection, model creation and training, and model evaluation phases. [Conclusion] We have made the final list of issues available to the community and believe it will help raise awareness about issues that need to be addressed throughout the ML life cycle to reduce TD and improve the maintainability of ML code.

Mon 28 Apr

Displayed time zone: Eastern Time (US & Canada) change

14:00 - 15:30
Quality Assurance for AI systemsResearch and Experience Papers at 208
Chair(s): Eduardo Santana de Almeida Federal University of Bahia
14:00
10m
Talk
Towards a Domain-Specific Modeling Language for Streamlined Change Management in AI Systems Development
Research and Experience Papers
Razan Abualsaud IRIT, CNRS, Toulouse
14:10
15m
Talk
An AI-driven Requirements Engineering Framework Tailored for Evaluating AI-Based Software
Research and Experience Papers
Hamed Barzamini , Fatemeh Nazaritiji Northern Illinois University, Annalise Brockmann Northern Illinois University, Hasan Ferdowsi Northern Illinois university, Mona Rahimi Northern Illinois University
14:25
15m
Talk
MLScent: A tool for Anti-pattern detection in ML projects
Research and Experience Papers
Karthik Shivashankar University of Oslo, Antonio Martini University of Oslo
14:40
15m
Talk
Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)Distinguished paper Award Candidate
Research and Experience Papers
Boyue Caroline Hu University of Toronto, Divya Gopinath KBR; NASA Ames, Ravi Mangal Colorado State University, Nina Narodytska VMware Research, Corina S. Păsăreanu Carnegie Mellon University, Susmit Jha SRI
14:55
15m
Talk
Investigating Issues that Lead to Code Technical Debt in Machine Learning Systems
Research and Experience Papers
Rodrigo Ximenes Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Antonio Pedro Santos Alves Pontifical Catholic University of Rio de Janeiro, Tatiana Escovedo Pontifical Catholic University of Rio de Janeiro, Rodrigo Spinola Virginia Commonwealth University, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio)
Pre-print
15:10
10m
Talk
Addressing Quality Challenges in Deep Learning: The Role of MLOps and Domain Knowledge
Research and Experience Papers
Santiago del Rey Universitat Politècnica De Catalunya - Barcelona Tech, Adrià Medina Universitat Politècnica de Barcelona - BarcelonaTech (UPC), Xavier Franch Universitat Politècnica de Catalunya, Silverio Martínez-Fernández UPC-BarcelonaTech
Pre-print
15:20
10m
Other
Discussion
Research and Experience Papers

:
:
:
: