Lessons Learned from Mining the Hugging Face Repository
The rapidly evolving fields of Machine Learning (ML) and Artificial Intelligence (AI) have witnessed the emergence of platforms like Hugging Face (HF) as central hubs for model development and sharing. This experience report synthesizes insights from two comprehensive studies conducted on HF, focusing on carbon emissions and the evolutionary and maintenance aspects of ML models. Our objective is to provide a practical guide for future researchers embarking on mining software repository studies within the HF ecosystem to enhance the quality of these studies. We delve into the intricacies of the replication package used in our studies, highlighting the pivotal tools and methodologies that facilitated our analysis. Furthermore, we propose a nuanced stratified sampling strategy tailored for the diverse HF Hub dataset, ensuring a representative and comprehensive analytical approach. The report also introduces preliminary guidelines, transitioning from repository mining to cohort studies, to establish causality in repository mining studies, particularly within the ML model of HF context. This transition is inspired by existing frameworks and is adapted to suit the unique characteristics of the HF model ecosystem. Our report serves as a guiding framework for researchers, contributing to the responsible and sustainable advancement of ML, and fostering a deeper understanding of the broader implications of ML models.
Tue 16 AprDisplayed time zone: Lisbon change
09:00 - 10:30 | Session 1 - Keynote & MSR StudiesWSESE at Eugénio de Andrade Chair(s): Andreas Jedlitschka Fraunhofer IESE | ||
09:00 15m | Welcome WSESE | ||
09:15 45mKeynote | Are we Getting Reliable Evidence? Methodology is Critical in Empirical Studies WSESE Natalia Juristo Universidad Politecnica de Madrid | ||
10:00 15mTalk | Lessons Learned from Mining the Hugging Face Repository WSESE Joel Castaño Fernández Universitat Politècnica de Catalunya, Silverio Martínez-Fernández UPC-BarcelonaTech, Xavier Franch Universitat Politècnica de Catalunya Link to publication Pre-print | ||
10:15 15mTalk | The Role of Data Filtering in Open Source Software Ranking and Selection WSESE Addi Malviya-Thakur The University of Tennessee, Knoxville / Oak Ridge National Laboratory, Audris Mockus The University of Tennessee, Knoxville / Vilnius University |