Backend support: Emma Söderberg

Abstract: Open science practices of open access publications, replication packages, and FAIR data (Findable, Accessible, Interoperable, Reusable) is gradually permeating empirical software engineering. Being a socio-technical research field, personal integrity must be balanced with the openness. Further, as collaboration with industry is an enabler for realism and relevance, industry needs of secrecy also must be balanced with transparency. In a recent paper, Runeson et al proposed a conceptual framework for dealing with decision on openness with respect to research data and artifacts. They derived the framework from one multi-method research project, involving a mixed-methods study with industry practitioners, design of a prototype code review tool, and evaluation in a user study, collecting eye-tracking data. The paper derives open science recommendations from this project. We aim to validate and extend the framework to more types of research artifacts and data.

Session Goals: The session aims at gathering the experiences of ISERN members in handling research data and artifacts, with respect to open science. We aim to validate the recommendations and extend the conceptual framework with more types of data and artifacts from the participants’ research projects.

Development of the Session: The session will be organized as a co-design workshop, where participants work in groups, map their research projects onto the conceptual framework, validate the framework elements and recommendations, and propose extensions needed.

• Introduction to the conceptual framework (15 mins)

• Working groups, mapping their research projects onto the framework (30 mins)

• Result presentation in plenary (30 mins)

• Summary of validation (15 mins)

What means for interaction will be used or required? Participants will need break out rooms and use paper sheets and post-it notes.

Background and recommended reading: Principal statement on open science in EMSE and the recently derived conceptual framework:

• Méndez Fernández, D., Monperrus, M., Feldt, R., Zimmermann, T., 2019. The open science initiative of the Empirical Software Engineering journal. Empirical Software Engineering 24, 1057–1060. https://doi.org/10.1007/s10664-019-09712-x.

• P. Runeson, E. Söderberg, and M. Höst. A conceptual framework and recommendations for open data and artifacts in empirical software engineering. In S. Vegas, A. Jedlitschka, and J. C. Carver, editors, 1st Int. Workshop on Methodological Issues with Empirical Studies in Software Engineering, ICSE Workshops, 2024. ACM. https://portal.research.lu.se/en/publications/a-conceptual-framework-and-recommendations-for-open-data-and-arti

Expected Outcomes and Plan for Continuing the Work beyond ISERN: We hope that the session will – through the validation and extension of the conceptual framework – elaborate practices for open science in empirical software engineering that are agreed in the community. We plan to publish the validation activity as a catalogue of projects mapped to the conceptual model.

Recommendations

R1. Give artifacts persistent DOI

R2. Make software open source with appropriate license and build communities

R3. Do not publish qualitative data, but study and analysis artifacts

R4. Share quantitative data openly, IFF sufficiently anonymized

R5. Do not open data which is traceable to individuals