Thu 26 Oct 2023 11:30 - 11:50 at Rhythms 3 - 1B - Machine learning in SE Chair(s): Davide Taibi

Background. Due to the widespread adoption of Artificial Intelligence (AI) and Machine Learning (ML) for building software applications, companies are struggling to recruit employees with a deep understanding of such technologies. In this scenario, AutoML is soaring as a promising solution to fill the AI/ML skills gap since it promises to automate the building of end-to-end AI/ML pipelines that would normally be engineered by specialized team members. Aims. Despite the growing interest and high expectations, there is a dearth of information about the extent to which AutoML is currently adopted by teams developing AI/ML-enabled systems and how it is perceived by practitioners and researchers. Method. To fill these gaps, in this paper, we present a mixed-method study comprising a benchmark of 12 end-to-end AutoML tools on two SE datasets and a user survey with follow-up interviews to further our understanding of AutoML adoption and perception. Results. We found that AutoML solutions can generate models that outperform those trained and optimized by researchers to perform classification tasks in the SE domain. Also, our findings show that the currently available AutoML solutions do not live up to their names as they do not equally support automation across the stages of the ML development workflow and for all the team members. Conclusions. We derive insights to inform the SE research community on how AutoML can facilitate their activities and tool builders on how to design the next generation of AutoML technologies.

Thu 26 Oct

Displayed time zone: Central Time (US & Canada) change

10:30 - 12:15
1B - Machine learning in SEESEM Technical Papers / ESEM Journal-First Papers / ESEM IGC at Rhythms 3
Chair(s): Davide Taibi University of Oulu
10:30
20m
Full-paper
What is the Carbon Footprint of ML Models on Hugging Face? A Repository Mining Study
ESEM Technical Papers
Joel Castaño Fernández Universitat Politècnica de Catalunya (UPC), Silverio Martínez-Fernández UPC-BarcelonaTech, Xavier Franch Universitat Politècnica de Catalunya, Justus Bogner Vrije Universiteit Amsterdam
Link to publication Pre-print
10:50
20m
Full-paper
An Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code
ESEM Technical Papers
Max Hort Simula Research Laboratory, Anastasiia Grishina Simula Research Laboratory, Leon Moonen Simula Research Laboratory and BI Norwegian Business School
Pre-print Media Attached
11:10
20m
Full-paper
An Empirical Study on Low- and High-Level Explanations of Deep Learning Misbehaviours
ESEM Technical Papers
Tahereh Zohdinasab USI Lugano, Vincenzo Riccio University of Udine, Paolo Tonella USI Lugano
11:30
20m
Full-paper
Assessing the Use of AutoML for Data-Driven Software Engineering
ESEM Technical Papers
Fabio Calefato University of Bari, Luigi Quaranta University of Bari, Italy, Filippo Lanubile University of Bari, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio)
Pre-print
11:50
10m
Journal Early-Feedback
An Empirical Study on ML DevOps Adoption Trends, Efforts, and Benefits Analysis
ESEM Journal-First Papers
Dhia Elhaq Rzig University of Michigan - Dearborn, Foyzul Hassan University of Michigan at Dearborn, Marouane Kessentini Oakland University
12:00
15m
Industry talk
The Perspective of Software Professionals on Algorithmic Racism
ESEM IGC