Understanding the Executability of Example Code in Hugging Face Model Cards: An Empirical Study
Thu 16 Apr 2026 13:30 - 14:00 at Catering and Exhibition Hall (Europa I to IV) - Poster Session 3 Chair(s): Rohit Gheyi, Grischa Liebel
Example code in Hugging Face model cards serves as an important entry point for users to understand and adopt pretrained models; however, during execution, users frequently encounter various errors, which in turn undermine reproducibility and practical usability. Given the long-standing lack of systematic evaluation of this phenomenon, we conduct a large-scale empirical study on the executability of model-card example code. Our results show that, among 291 model cards, 65.98% of example code fails to execute successfully, indicating substantial limitations in real-world usability. Further analysis reveals that failure rates increase with model size and vary significantly across task types, with speech, vision, multimodal, and generative tasks being particularly fragile. Based on execution logs, we identify three dominant root causes: insufficient documentation of dependency and version requirements, implicit runtime environment assumptions embedded in example code, and incompatibilities induced by ecosystem evolution (e.g., API changes). This study provides a preliminary empirical evidence for understanding the effectiveness of model-card examples in real-world usage scenarios.
Wed 15 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
15:30 - 16:00 | Poster Session 2Posters at Catering and Exhibition Hall (Europa I to IV) Chair(s): Rohit Gheyi Federal University of Campina Grande, Grischa Liebel Reykjavik University | ||
15:30 30mPoster | Applying Neuroinclusive Agile Practices in Software Engineering Team-Based Learning Posters Cristiana Lacerda Universidade Federal de Pernambuco / CESAR School, Kiev Gama Universidade Federal de Pernambuco, Aline Lacerda Federal University of Pernambuco (UFPE) | ||
15:30 30mPoster | Too Hot To Handle: The Effects of Temperature on AI-Generated Code Used to Cheat in Programming Assignments Posters Robin Maisch Karlsruhe Institute of Technology (KIT), Timur Sağlam Karlsruhe Institute of Technology (KIT), Larissa Schmid KTH Royal Institute of Technology, Sebastian Hahner Karlsruhe Institute of Technology (KIT), Tobias Hey Karlsruhe Institute of Technology (KIT) | ||
15:30 30mPoster | Understanding the Executability of Example Code in Hugging Face Model Cards: An Empirical Study Posters Simeng Yao National University of Defense Technology, Jialin Zhao National University of Defense Technology, Yang Zhang National University of Defense Technology, China, Yiwen Wu National University of Defense Technology, Tao Wang National University of Defense Technology, Xunhui Zhang National University of Defense Technology, China | ||
15:30 30mPoster | Towards Understanding Views on Combining Videos and Gamification in Software Engineering Training Posters Pasan Peiris University of Canterbury, New Zealand, Matthias Galster University of Canterbury, Antonija Mitrovic University of Canterbury, Sanna Malinen University of Canterbury, Raul Vincent Lumapas University of Canterbury, New Zealand, Jay Holland University of Canterbury | ||
15:30 30mPoster | Specification Generation for Neural Networks in Systems Posters Isha Chaudhary University of Illinois, Urbana-Champaign, Shuyi Lin Northeastern University, Cheng Tan Northeastern University, Gagandeep Singh University of Illinois Urbana-Champaign | ||
15:30 30mPoster | Ethicode City: Gamifying Software Engineering Ethics Education Posters Shivadharshan S Indian Institute of Technology Tirupati, Akilesh P Indian Institute of Technology Tirupati, Sridhar Chimalakonda Indian Institute of Technology Tirupati | ||
15:30 30mPoster | Rate My Web Design? Using LLMs to Predict Human Subjective Ratings of Generated Web UI Prototypes Posters | ||
15:30 30mPoster | Modular Workshop Architecture for Teaching Machine Learning in Autonomous Driving by Non-Experts Posters Lukas Walter Hamm-Lippstadt University of Applied Sciences, Christopher Beck Hamm-Lippstadt University of Applied Sciences, Stefan Henkler Hochschule Hamm-Lippstadt | ||
15:30 30mPoster | Unsupervised Approaches to Futile Cycle Detection in AI Agents Posters Felix George , Harshit Kumar IBM Research, Divya Pathak IBM Research - India, Kaustabha Ray IBM Research - India, Mudit Verma IBM Research - India, Pratibha Moogi IBM Research - India | ||
15:30 30mPoster | The Energy Consumption of Legacy COBOL Systems: A Forgotten Direction and COBJoules Tool Posters Shivadharshan S Indian Institute of Technology Tirupati, Sridhar Chimalakonda Indian Institute of Technology Tirupati | ||
15:30 30mPoster | Textbook-Embedded Virtual Teaching Assistant That Answers CS1 Questions With Verifiable Citations Posters | ||
15:30 30mPoster | Games Designed to Teach or Practice Technical Debt Management: A Systematic Mapping Study Posters Mahsa Radnejad University of Maryland Baltimore County, Carolyn Seaman University of Maryland Baltimore County | ||
15:30 30mPoster | Teaching Emotional Awareness in Software Engineering: Evidence from a Professional Training Program Posters Daniela Grassi University of Bari, Filippo Lanubile University of Bari, Nicole Novielli University of Bari, Alexander Serebrenik Eindhoven University of Technology | ||
15:30 30mPoster | Anxiety-Aware Requirements Engineering: A Vision Paper Posters Sofia Ouhbi Uppsala University | ||
15:30 30mPoster | From Empathy to Engineering: A Pedagogical Framework and its Authentic mHealth Implementation Posters Wei Wang Institute of Software at Chinese Academy of Sciences, Jialong Li Waseda University, Japan, Devi Karolita University of Palangka Raya | ||
15:30 30mPoster | Exploring the Information Disorientation Problem in Automated Program Repair under Relaxed Localization Posters Jinrui Sun Nankai University, Tong Jia Institute for Artificial Intelligence, Peking University, Beijing, China, Minghua He Peking University, Ying Li School of Software and Microelectronics, Peking University, Beijing, China | ||
15:30 30mPoster | LLM-Native Applications: Toward a New Paradigm in Software Engineering Posters Alan Bandeira State University of Ceará - UECE, Gabriel Luiz Barros De Oliveira State University of Ceará - UECE, Paulo Maia State University of Ceará, Carlos Aderaldo University of Fortaleza, Nabor Mendonca University of Fortaleza | ||
Thu 16 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
13:30 - 14:00 | Poster Session 3Posters at Catering and Exhibition Hall (Europa I to IV) Chair(s): Rohit Gheyi Federal University of Campina Grande, Grischa Liebel Reykjavik University | ||
13:30 30mPoster | LLM-Native Applications: Toward a New Paradigm in Software Engineering Posters Alan Bandeira State University of Ceará - UECE, Gabriel Luiz Barros De Oliveira State University of Ceará - UECE, Paulo Maia State University of Ceará, Carlos Aderaldo University of Fortaleza, Nabor Mendonca University of Fortaleza | ||
13:30 30mPoster | Unsupervised Approaches to Futile Cycle Detection in AI Agents Posters Felix George , Harshit Kumar IBM Research, Divya Pathak IBM Research - India, Kaustabha Ray IBM Research - India, Mudit Verma IBM Research - India, Pratibha Moogi IBM Research - India | ||
13:30 30mPoster | Textbook-Embedded Virtual Teaching Assistant That Answers CS1 Questions With Verifiable Citations Posters | ||
13:30 30mPoster | Towards Understanding Views on Combining Videos and Gamification in Software Engineering Training Posters Pasan Peiris University of Canterbury, New Zealand, Matthias Galster University of Canterbury, Antonija Mitrovic University of Canterbury, Sanna Malinen University of Canterbury, Raul Vincent Lumapas University of Canterbury, New Zealand, Jay Holland University of Canterbury | ||
13:30 30mPoster | From Empathy to Engineering: A Pedagogical Framework and its Authentic mHealth Implementation Posters Wei Wang Institute of Software at Chinese Academy of Sciences, Jialong Li Waseda University, Japan, Devi Karolita University of Palangka Raya | ||
13:30 30mPoster | Applying Neuroinclusive Agile Practices in Software Engineering Team-Based Learning Posters Cristiana Lacerda Universidade Federal de Pernambuco / CESAR School, Kiev Gama Universidade Federal de Pernambuco, Aline Lacerda Federal University of Pernambuco (UFPE) | ||
13:30 30mPoster | Specification Generation for Neural Networks in Systems Posters Isha Chaudhary University of Illinois, Urbana-Champaign, Shuyi Lin Northeastern University, Cheng Tan Northeastern University, Gagandeep Singh University of Illinois Urbana-Champaign | ||
13:30 30mPoster | Modular Workshop Architecture for Teaching Machine Learning in Autonomous Driving by Non-Experts Posters Lukas Walter Hamm-Lippstadt University of Applied Sciences, Christopher Beck Hamm-Lippstadt University of Applied Sciences, Stefan Henkler Hochschule Hamm-Lippstadt | ||
13:30 30mPoster | Teaching Emotional Awareness in Software Engineering: Evidence from a Professional Training Program Posters Daniela Grassi University of Bari, Filippo Lanubile University of Bari, Nicole Novielli University of Bari, Alexander Serebrenik Eindhoven University of Technology | ||
13:30 30mPoster | The Energy Consumption of Legacy COBOL Systems: A Forgotten Direction and COBJoules Tool Posters Shivadharshan S Indian Institute of Technology Tirupati, Sridhar Chimalakonda Indian Institute of Technology Tirupati | ||
13:30 30mPoster | Anxiety-Aware Requirements Engineering: A Vision Paper Posters Sofia Ouhbi Uppsala University | ||
13:30 30mPoster | Understanding the Executability of Example Code in Hugging Face Model Cards: An Empirical Study Posters Simeng Yao National University of Defense Technology, Jialin Zhao National University of Defense Technology, Yang Zhang National University of Defense Technology, China, Yiwen Wu National University of Defense Technology, Tao Wang National University of Defense Technology, Xunhui Zhang National University of Defense Technology, China | ||
13:30 30mPoster | Rate My Web Design? Using LLMs to Predict Human Subjective Ratings of Generated Web UI Prototypes Posters | ||
13:30 30mPoster | Too Hot To Handle: The Effects of Temperature on AI-Generated Code Used to Cheat in Programming Assignments Posters Robin Maisch Karlsruhe Institute of Technology (KIT), Timur Sağlam Karlsruhe Institute of Technology (KIT), Larissa Schmid KTH Royal Institute of Technology, Sebastian Hahner Karlsruhe Institute of Technology (KIT), Tobias Hey Karlsruhe Institute of Technology (KIT) | ||
13:30 30mPoster | Exploring the Information Disorientation Problem in Automated Program Repair under Relaxed Localization Posters Jinrui Sun Nankai University, Tong Jia Institute for Artificial Intelligence, Peking University, Beijing, China, Minghua He Peking University, Ying Li School of Software and Microelectronics, Peking University, Beijing, China | ||
13:30 30mPoster | Games Designed to Teach or Practice Technical Debt Management: A Systematic Mapping Study Posters Mahsa Radnejad University of Maryland Baltimore County, Carolyn Seaman University of Maryland Baltimore County | ||
13:30 30mPoster | Ethicode City: Gamifying Software Engineering Ethics Education Posters Shivadharshan S Indian Institute of Technology Tirupati, Akilesh P Indian Institute of Technology Tirupati, Sridhar Chimalakonda Indian Institute of Technology Tirupati | ||