Write a Blog >>
ICSE 2021
Mon 17 May - Sat 5 June 2021
Tue 25 May 2021 15:40 - 16:00 at Blended Sessions Room 3 - 1.3.3. Open Science Chair(s): Sira Vegas
Wed 26 May 2021 03:40 - 04:00 at Blended Sessions Room 3 - 1.3.3. Open Science

Open-science initiatives have gained substantial momentum in computer science, and particularly in software-engineering research. A critical aspect of open-science is the public availability of artifacts (e.g., tools), which facilitates the replication, reproduction, extension, and verification of results. While we experienced that many artifacts are not publicly available, we are not aware of empirical evidence supporting this subjective claim. In our article, we report an empirical study on software artifact papers (SAPs) published at the International Conference on Software Engineering (ICSE), in which we investigated whether and how researchers have published their software artifacts, and whether this had scientific impact. Our dataset comprises 789 ICSE research track papers, including 604 SAPs (76.6%), from the years 2007 to 2017. While showing a positive trend towards artifact availability, our results are still sobering. Even in 2017, only 58.5% of the papers that stated to have developed a software artifact made that artifact publicly available. As we did find a small, but statistically significant, positive correlation between linking to artifacts in a paper and its scientific impact in terms of citations, we hope to motivate the research community to share more artifacts. With our insights, we aim to support the advancement of open science by discussing our results in the context of existing initiatives and guidelines. In particular, our findings advocate the need for clearly communicating artifacts and the use of non-commercial, persistent archives to provide replication packages.

Conference Day
Tue 25 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

15:20 - 16:15
1.3.3. Open ScienceNIER - New Ideas and Emerging Results / Journal-First Papers at Blended Sessions Room 3 +12h
Chair(s): Sira VegasUniversidad Politecnica de Madrid
15:20
20m
Paper
What Makes a Popular Academic AI Repository?Journal-First
Journal-First Papers
Yuanrui FanCollege of Computer Science and Technology, Zhejiang University, Xin XiaHuawei Software Engineering Application Technology Lab, David LoSingapore Management University, Ahmed E. HassanSchool of Computing, Queen's University, Shanping LiZhejiang University
Pre-print Media Attached
15:40
20m
Paper
Publish or Perish, but do not Forget your Software ArtifactsJournal-First
Journal-First Papers
Robert HeumüllerChair of Software Engineering, Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, Sebastian NielebockOtto-von-Guericke University Magdeburg, Germany, Jacob KrügerOtto von Guericke University Magdeburg, Frank OrtmeierOtto-von-Guericke-University Magdeburg, Faculty of Computer Science, Chair of Software Engineering
DOI Pre-print Media Attached
16:00
15m
Paper
Stop Building Castles on a Swamp! The Crisis of Reproducing Automatic Search in Evidence-based Software EngineeringNIER
NIER - New Ideas and Emerging Results
Zheng LiUniversity of Concepción
Pre-print Media Attached

Conference Day
Wed 26 May

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

03:20 - 04:15
03:20
20m
Paper
What Makes a Popular Academic AI Repository?Journal-First
Journal-First Papers
Yuanrui FanCollege of Computer Science and Technology, Zhejiang University, Xin XiaHuawei Software Engineering Application Technology Lab, David LoSingapore Management University, Ahmed E. HassanSchool of Computing, Queen's University, Shanping LiZhejiang University
Pre-print Media Attached
03:40
20m
Paper
Publish or Perish, but do not Forget your Software ArtifactsJournal-First
Journal-First Papers
Robert HeumüllerChair of Software Engineering, Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, Sebastian NielebockOtto-von-Guericke University Magdeburg, Germany, Jacob KrügerOtto von Guericke University Magdeburg, Frank OrtmeierOtto-von-Guericke-University Magdeburg, Faculty of Computer Science, Chair of Software Engineering
DOI Pre-print Media Attached
04:00
15m
Paper
Stop Building Castles on a Swamp! The Crisis of Reproducing Automatic Search in Evidence-based Software EngineeringNIER
NIER - New Ideas and Emerging Results
Zheng LiUniversity of Concepción
Pre-print Media Attached