ICSE 2024
Fri 12 - Sun 21 April 2024 Lisbon, Portugal

The rapidly evolving fields of Machine Learning (ML) and Artificial Intelligence (AI) have witnessed the emergence of platforms like Hugging Face (HF) as central hubs for model development and sharing. This experience report synthesizes insights from two comprehensive studies conducted on HF, focusing on carbon emissions and the evolutionary and maintenance aspects of ML models. Our objective is to provide a practical guide for future researchers embarking on mining software repository studies within the HF ecosystem to enhance the quality of these studies. We delve into the intricacies of the replication package used in our studies, highlighting the pivotal tools and methodologies that facilitated our analysis. Furthermore, we propose a nuanced stratified sampling strategy tailored for the diverse HF Hub dataset, ensuring a representative and comprehensive analytical approach. The report also introduces preliminary guidelines, transitioning from repository mining to cohort studies, to establish causality in repository mining studies, particularly within the ML model of HF context. This transition is inspired by existing frameworks and is adapted to suit the unique characteristics of the HF model ecosystem. Our report serves as a guiding framework for researchers, contributing to the responsible and sustainable advancement of ML, and fostering a deeper understanding of the broader implications of ML models.

Tue 16 Apr

Displayed time zone: Lisbon change

09:00 - 10:30
Session 1 - Keynote & MSR StudiesWSESE at Eugénio de Andrade
Chair(s): Andreas Jedlitschka Fraunhofer IESE
09:00
15m
Welcome
WSESE
Sira Vegas Universidad Politecnica de Madrid, Andreas Jedlitschka Fraunhofer IESE
09:15
45m
Keynote
Are we Getting Reliable Evidence? Methodology is Critical in Empirical Studies
WSESE
Natalia Juristo Universidad Politecnica de Madrid
10:00
15m
Talk
Lessons Learned from Mining the Hugging Face Repository
WSESE
Joel Castaño Fernández Universitat Politècnica de Catalunya, Silverio Martínez-Fernández UPC-BarcelonaTech, Xavier Franch Universitat Politècnica de Catalunya
Link to publication Pre-print
10:15
15m
Talk
The Role of Data Filtering in Open Source Software Ranking and Selection
WSESE
Addi Malviya-Thakur The University of Tennessee, Knoxville / Oak Ridge National Laboratory, Audris Mockus The University of Tennessee, Knoxville / Vilnius University