OpenART: Manuscript-Driven Support for Promoting Executability of Software Engineering Artifacts
Executability of replication packages is a prerequisite for validating, reusing, and extending results in software engineering (SE) research. Despite substantial progress in artifact availability and badging, recent large-scale studies reveal that a majority of published replication packages remain partially executable or fail entirely, most often due to incomplete documentation, missing components, and unclear execution procedures. These issues impose a significant burden on reviewers and secondary researchers, undermining the reusability and long-term value of research artifacts. This paper introduces OpenART, a framework designed to systematically improve the completeness of SE replication packages to enhance executability. OpenART addresses a critical gap in current artifact practices by automatically deriving actionable artifact specifications directly from companion manuscripts. Leveraging both foundation and fine-tuned large language models, OpenART extracts explicit and implicit execution-relevant information from papers to generate structured artifact documentation, including required components, dependencies, and execution steps. From this documentation, OpenART further derives an expected artifact layout that enables (i) guided preparation of replication packages from scratch and (ii) automated identification of missing or inconsistent components in existing packages. We evaluate OpenART through a mixed-method user study with SE researchers across two realistic workflows: artifact creation and artifact completion. We expect that OpenART will substantially reduce preparation effort, improve perceived completeness, and enhance confidence in the executability of artifacts. OpenART will be the first framework to demonstrate the feasibility of manuscript-driven, automated artifact support and to offer a concrete step toward scalable, reviewer-friendly artifact evaluation in SE research.
Tue 14 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
14:00 - 15:30 | |||
14:00 5mTalk | Is Architecting Still a Human Activity? Journal Ahead Workshop (JAWs) Domenico Amalfitano University of Naples Federico II, Marco Autili University of L'Aquila, Italy, Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy | ||
14:05 5mTalk | FormIDEAble: Safe and Socially-aware Autonomous Systems Journal Ahead Workshop (JAWs) Livia Lestingi DEIB, Politecnico di Milano, Amel Bennaceur The Open University, UK, Marcello M. Bersani Politecnico di Milano, Carlos Gavidia-Calderon The Alan Turing Institute, Anastasia Kordoni Lancaster University (UK), Mark Levine Lancaster University, Bashar Nuseibeh The Open University, UK; Lero, University of Limerick, Ireland, Matteo Rossi Politecnico di Milano | ||
14:10 5mTalk | The Runtime Dimension of Ethics in Self-Adaptive Systems Journal Ahead Workshop (JAWs) Marco Autili University of L'Aquila, Italy, Gianluca Filippone Gran Sasso Science Institute, L'Aquila, Italy, Mashal Afzal Memon University of L’Aquila, Italy, Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy | ||
14:15 5mTalk | Evaluating RE Practices for Explainability: Synthesizing Insights from Daimler Truck into an Explainable RE Framework Proposal Journal Ahead Workshop (JAWs) Umm-e Habiba Technical University of Munich, Lucas Mauser Daimler Truck AG / Technical University of Munich, Jonas Fritzsch University of Stuttgart, Institute of Software Engineering, Justus Bogner Vrije Universiteit Amsterdam, Stefan Wagner Technical University of Munich | ||
14:20 5mTalk | Bridging AI Ethics and Requirements Engineering in Healthcare: Triangulating Evidence from Literature, Experts, and Online Communities Journal Ahead Workshop (JAWs) Yutan Huang Monash University, Chetan Arora Monash University, Anuradha Madugalla Deakin University, School of IT, Australia, Tanjila Kanij , Fanyu Wang Monash University, John Grundy Monash University | ||
14:25 5mTalk | Future of Agile: A Multiple-Case Study of AI-Native Software Development Journal Ahead Workshop (JAWs) | ||
14:30 5mTalk | Weaponizing the Commons: A Taxonomy and Detection Framework of Abuse on GitHub Journal Ahead Workshop (JAWs) Yuli Cheng Xi'an Jiaotong University: Xi'an, Shaanxi, CN, Xiaoyu Zhang Nanyang Technological University, Singapore, Jiongchi Yu Singapore Management University, Shiqing Ma University of Massachusetts at Amherst, Chao Shen Xi’an Jiaotong University, Yang Liu Nanyang Technological University | ||
14:35 5mTalk | Do Socially Close Developers Clone More? Tracing the Link Between Collaboration and Duplication Journal Ahead Workshop (JAWs) Palash Ranjan Roy University of Saskatchewan, Zixian Zhang School of Computer Science, University of Galway, Julita Vassileva University of Saskatchewan, Banani Roy University of Saskatchewan, Kevin Schneider University of Saskatchewan, Chanchal K. Roy University of Saskatchewan, Takfarinas Saber University of Galway | ||
14:40 5mTalk | OpenART: Manuscript-Driven Support for Promoting Executability of Software Engineering Artifacts Journal Ahead Workshop (JAWs) Al Muttakin University of Saskatchewan, Saikat Mondal University of Saskatchewan, Chanchal K. Roy University of Saskatchewan | ||
14:45 5mTalk | From Generic to Personalized: Exploring Persona-Aware Code Review Explanations Journal Ahead Workshop (JAWs) Shamse Tasnim Cynthia University of Saskatchewan, Ratnadira Widyasari Singapore Management University, Singapore, Banani Roy University of Saskatchewan, Italo Santos University of Hawai‘i at Mānoa, David Lo Singapore Management University | ||
14:50 5mTalk | From Copy-Paste to Correct: Governing the Transfer of Software Practices Journal Ahead Workshop (JAWs) Ajmain Inqiad Alam University of Saskatchewan, Chanchal K. Roy University of Saskatchewan, Banani Roy University of Saskatchewan, Kevin Schneider University of Saskatchewan | ||
14:55 25mPanel | Panel Discussion: Human and Social Factors Journal Ahead Workshop (JAWs) | ||
15:20 10mAwards | Selection of the award presentations Journal Ahead Workshop (JAWs) | ||