Write a Blog >>
ICSE 2021
Sun 16 May - Sat 5 June 2021

This program is tentative and subject to change.

Tue 25 May 2021 15:40 - 16:00 at Blended Sessions Room 3 - 1.3.3. Open Science Chair(s): Sira Vegas

Open-science initiatives have gained substantial momentum in computer science, and particularly in software-engineering research. A critical aspect of open-science is the public availability of artifacts (e.g., tools), which facilitates the replication, reproduction, extension, and verification of results. While we experienced that many artifacts are not publicly available, we are not aware of empirical evidence supporting this subjective claim. In our article, we report an empirical study on software artifact papers (SAPs) published at the International Conference on Software Engineering (ICSE), in which we investigated whether and how researchers have published their software artifacts, and whether this had scientific impact. Our dataset comprises 789 ICSE research track papers, including 604 SAPs (76.6%), from the years 2007 to 2017. While showing a positive trend towards artifact availability, our results are still sobering. Even in 2017, only 58.5% of the papers that stated to have developed a software artifact made that artifact publicly available. As we did find a small, but statistically significant, positive correlation between linking to artifacts in a paper and its scientific impact in terms of citations, we hope to motivate the research community to share more artifacts. With our insights, we aim to support the advancement of open science by discussing our results in the context of existing initiatives and guidelines. In particular, our findings advocate the need for clearly communicating artifacts and the use of non-commercial, persistent archives to provide replication packages.

This program is tentative and subject to change.

Tue 25 May
Times are displayed in time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

15:20 - 16:15
1.3.3. Open ScienceTechnical Track / NIER - New Ideas and Emerging Results / Journal-First Papers at Blended Sessions Room 3
Chair(s): Sira VegasUniversidad Politecnica de Madrid
15:20
20m
Paper
What Makes a Popular Academic AI Repository?Journal-First
Journal-First Papers
Yuanrui FanCollege of Computer Science and Technology, Zhejiang University, Xin XiaMonash University, David LoSingapore Management University, Ahmed E. HassanSchool of Computing, Queen's University, Shanping LiZhejiang University
Pre-print
15:40
20m
Paper
Publish or Perish, but do not Forget your Software ArtifactsJournal-First
Journal-First Papers
Robert HeumüllerChair of Software Engineering, Faculty of Computer Science, Otto-von-Guericke-University Magdeburg, Sebastian NielebockOtto-von-Guericke University Magdeburg, Germany, Jacob KrügerOtto von Guericke University Magdeburg, Frank OrtmeierOtto-von-Guericke-University Magdeburg, Faculty of Computer Science, Chair of Software Engineering
DOI Pre-print Media Attached
16:00
15m
Paper
Stop Building Castles on a Swamp! The Crisis of Reproducing Automatic Search in Evidence-based Software EngineeringNIER
NIER - New Ideas and Emerging Results
Zheng LiUniversity of Concepción
Pre-print