Requirements Quality Research Artifacts: Recovery, Analysis, and Management Guideline
Requirements quality research produces methods and tools to assess and improve the quality of requirements specifications. Core contributions to this research area are research artifacts, like data sets and implementations. The former provide benchmarks for requirements quality, while the latter automate the detection and removal of defects. However, research artifacts are prone to become unavailable or are not disclosed by their authors in the first place. This inhibits progress in the research community as it renders the reuse of artifacts impossible. In this work, we aimed to improve the availability of research artifacts in requirements quality research, both retrospectively and prospectively. For retrospective improvement, we (1) conducted an artifact recovery initiative, where we guided authors in making their artifacts available again. For prospective improvement, we (2) empirically evaluated the reasons for artifact unavailability, and (3) compiled concise guidelines for open science artifacts. Our results include 10 recovered data sets and 7 recovered implementations that were previously unavailable. Our empirical analysis supports that artifact availability improved over time and that public hosting services positively affect artifact longevity. Finally, our pragmatic open science guidelines compile advice into an actionable process. With this work, we hope to encourage and support adherence to open science principles and improve the availability of research artifacts for the requirements quality research community.