Publish or Perish, but do not Forget your Software ArtifactsJournal-First
This program is tentative and subject to change.
Open-science initiatives have gained substantial momentum in computer science, and particularly in software-engineering research. A critical aspect of open-science is the public availability of artifacts (e.g., tools), which facilitates the replication, reproduction, extension, and verification of results. While we experienced that many artifacts are not publicly available, we are not aware of empirical evidence supporting this subjective claim. In our article, we report an empirical study on software artifact papers (SAPs) published at the International Conference on Software Engineering (ICSE), in which we investigated whether and how researchers have published their software artifacts, and whether this had scientific impact. Our dataset comprises 789 ICSE research track papers, including 604 SAPs (76.6%), from the years 2007 to 2017. While showing a positive trend towards artifact availability, our results are still sobering. Even in 2017, only 58.5% of the papers that stated to have developed a software artifact made that artifact publicly available. As we did find a small, but statistically significant, positive correlation between linking to artifacts in a paper and its scientific impact in terms of citations, we hope to motivate the research community to share more artifacts. With our insights, we aim to support the advancement of open science by discussing our results in the context of existing initiatives and guidelines. In particular, our findings advocate the need for clearly communicating artifacts and the use of non-commercial, persistent archives to provide replication packages.
This program is tentative and subject to change.
Tue 25 May Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
15:20 - 16:15 | 1.3.3. Open ScienceTechnical Track / NIER - New Ideas and Emerging Results / Journal-First Papers at Blended Sessions Room 3 Chair(s): Sira VegasUniversidad Politecnica de Madrid | ||
15:20 20mPaper | What Makes a Popular Academic AI Repository?Journal-First Journal-First Papers Yuanrui FanCollege of Computer Science and Technology, Zhejiang University, Xin XiaMonash University, David LoSingapore Management University, Ahmed E. HassanSchool of Computing, Queen's University, Shanping LiZhejiang University Pre-print | ||
15:40 20mPaper | Publish or Perish, but do not Forget your Software ArtifactsJournal-First Journal-First Papers Robert HeumüllerChair of Software Engineering, Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, Sebastian NielebockOtto-von-Guericke University Magdeburg, Germany, Jacob KrügerOtto von Guericke University Magdeburg, Frank OrtmeierOtto-von-Guericke-University Magdeburg, Faculty of Computer Science, Chair of Software Engineering DOI Pre-print Media Attached | ||
16:00 15mPaper | Stop Building Castles on a Swamp! The Crisis of Reproducing Automatic Search in Evidence-based Software EngineeringNIER NIER - New Ideas and Emerging Results Zheng LiUniversity of Concepción Pre-print |