Aurora B is the second room in the Aurora wing. When facing the main Cosmos Hall, access to the Aurora wing is on the right, close to the side entrance of the hotel.
Aurora B is the second room in the Aurora wing.
When facing the main Cosmos Hall, access to the Aurora wing is on the right, close to the side entrance of the hotel.
Over the past decades, reports of reproducibility crises have surfaced in various scientific communities. Independent confirmations of published research results failed, casting doubt about the validity of these results. Even before the magnitude of the problem has become apparent in many domains, the software-engineering community introduced artifact evaluations, for the first time at ESEC/FSE 2011, in which research artifacts that support published results were voluntarily submitted for peer review. Since then, artifact evaluations have become immensely popular and are today being offered to authors at most software-engineering venues, where large artifact-evaluation committees handle large numbers of artifact submissions.
To make sure that this enormous effort from our community to (a) create and (b) assess research artifacts is well-spent, knowledge and insights from successful and unsuccessful artifact-evaluation practices as well as publishing implications need to be conserved and shared with prospective participants, i.e., authors, reviewers, and organizers. Based on insights from empirical studies about artifact evaluations in the software-engineering community, from running artifact evaluations at different conferences, and from publishing experience at Conference Publishing Consulting, this tutorial presents an overview what artifact evaluations are and how they are conducted, along with known pitfalls and established best practices to overcome them. The presented insights will be accompanied by a hands-on training session on artifact evaluation using published research artifacts. The tutorial targets prospective artifact-evaluation organizers and reviewers as well as researchers wishing to improve the utility of the research artifacts they create.
Speakers:
Participant Requirements: Working laptop with Internet access
Material:
Duration: 180 minutes
Time plan: