PROFES 2023
Sun 10 - Wed 13 December 2023 Dornbirn, Austria
Tue 12 Dec 2023 14:10 - 14:20 at W211 - Machine Learning and Data Science Chair(s): Xiaozhou Li

Systems that use Machine Learning (ML) have become commonplace for companies that want to improve their products and processes. Literature suggests that Requirements Engineering (RE) can help address many problems when engineering ML-enabled systems. However, the state of empirical evidence on how RE is applied in practice in the context of ML-enabled systems is mainly dominated by isolated case studies with limited generalizability. We conducted an international survey to gather practitioner insights into the status quo and problems of RE in ML-enabled systems. We gathered 188 complete responses from 25 countries. We conducted quantitative statistical analyses on contemporary practices using bootstrapping with confidence intervals and qualitative analyses on the reported problems involving open and axial coding procedures. We found significant differences in RE practices within ML projects. For instance, (i) RE-related activities are mostly conducted by project leaders and data scientists, (ii) the prevalent requirements documentation format concerns interactive Notebooks, (iii) the main focus of non-functional requirements includes data quality, model reliability, and model explainability, and (iv) main challenges include managing customer expectations and aligning requirements with data. The qualitative analyses revealed that practitioners face problems related to lack of business domain understanding, unclear goals and requirements, low customer engagement, and communication issues. These results help to provide a better understanding of the adopted practices and of which problems exist in practical environments. We put forward the need to adapt further and disseminate RE-related practices for engineering ML-enabled systems.

Tue 12 Dec

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

14:00 - 15:00
Machine Learning and Data ScienceResearch Papers / Organization / Short Papers and Posters / Industry Papers at W211
Chair(s): Xiaozhou Li University of Oulu
14:00
10m
Short-paper
Operationalizing Assurance Cases for Data Scientists: A Showcase of Concepts and Tooling in the Context of Test Data Quality for Machine Learning
Short Papers and Posters
Lisa Jöckel Fraunhofer, Michael Klaes Fraunhofer IESE; Konstanz University of Applied Sciences, Janek Groß Fraunhofer IESE, Pascal Gerber Fraunhofer Institute for Experimental Software Engineering, Markus Scholz NovelSense, Jonathan Eberle TRUMPF, Marc Teschner TRUMPF, Daniel Seifert Fraunhofer IESE, Richard Hawkins University of York, John Molloy University of York, Jens Ottnad TRUMPF
14:10
10m
Research paper
Status Quo and Problems of Requirements Engineering for Machine Learning: Results from an International Survey
Research Papers
Antonio Pedro Santos Alves Pontifical Catholic University of Rio de Janeiro, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Görkem Giray Independent Researcher, Daniel Mendez Blekinge Institute of Technology, Niklas Lavesson Blekinge Institute of Technology, Kelly Azevedo Pontifical Catholic University of Rio de Janeiro, Hugo Villamizar Pontifical Catholic University of Rio de Janeiro (PUC-Rio), Tatiana Escovedo Pontifical Catholic University of Rio de Janeiro, Helio Côrtes Vieira Lopes PUC-Rio, Stefan Biffl Vienna University of Technology, Juergen Musil , Michael Felderer German Aerospace Center (DLR) & University of Cologne, Stefan Wagner University of Stuttgart, Maria Teresa Baldassarre Department of Computer Science, University of Bari , Tony Gorschek Blekinge Institute of Technology / DocEngineering
14:20
10m
Short-paper
A Stochastic Approach Based on Rational Decision-Making for Analyzing Software Engineering Project Status
Short Papers and Posters
Hannes Salin Dalarna University
14:30
10m
Research paper
CAIS-DMA: A Decision-Making Assistant for Collaborative AI Systems
Research Papers
Diaeddin Rimawi Free University of Bozen-Bolzano, Antonio Liotta , Marco Todescato , Barbara Russo
14:40
10m
Short-paper
Comparing machine learning algorithms for medical time-series data
Short Papers and Posters
Aliasgar Shereef Chalmers University of Technology, Alex Helmersson Chalmers University of Technology, Faton Hoti Chalmers University of Technology, Sebastian Levander Gothenburg University, Emil Svensson Chalmers University of Technology, Ali El-Merhi Sahlgrenska University Hospital, Richard Vithal Sahlgrenska University Hospital, Jaquette Liljenqrantz Sahlgrenska University Hospital, Linda Block Sahlgrenska University Hospital , Helena Odenstedt Sahlgrenska University Hospital, Miroslaw Staron Chalmers University of Technology
14:50
10m
Research paper
What Data Scientists (Care to) Recall
Research Papers
Samar Saeed University of Gothenburg, Shahrzad Sheikholeslami University of Gothenburg, Jacob Krüger Eindhoven University of Technology, Regina Hebig University of Rostock