Abstract

Empirical research in software engineering often faces the problem that contributions to one research endeavor (e.g., test-driven development) are disconnected from each other in terms of relation to existing evidence, they are often heterogeneous, and studies often seem uncoordinated. This renders the synthesis of individual empirical studies into more robust evidence cumbersome, and it impedes productive, distributed research programs to arrive at conclusions with a strong external validity which would be desirable for knowledge translation. We postulate that we need a more structured approach that allows us to effectively synthesize evidence and increase the robustness of our scientific theories in the long run.

To address this issue, we developed an initial prototype for a framework for managing scientific theories. Our ambition is to establish a community platform that allows for a “version-control” of hypotheses and, in consequence, evidence. We plan to implement this framework into a publicly accessible system that allows our research community to facilitate distributed, constructive, and coherent research endeavors. We cordially invite the ISERN community to the co-development of this framework to ensure its long-term usability before finalizing the framework.

Session Goal

The goals of this upcoming thematic workshop are to:

1. Familiarize participants with the initiative for managing scientific SE theories.

2. Pooling the experiences and expertise of ISERN members of developing and maintaining scientific SE theories to make sure that the framework is usable for the empirical SE community.

3. Opening the initiative up for contributions from the ESEM community in the form of active collaborations.

Development of the Session

To conduct the workshop in a systematic manner, we propose to:

1. Introduce the current state of the framework as well as the envisioned implementation of it.

2. Discuss the applicability of the framework in an open discussion with all participants.

3. Form interactive focus groups to discuss one a particularly critical aspect of the framework, the measurement of latent context variables.

4. Outline our plans for developing and maintaining a system implementing the platform and inviting participants to contribute to this endeavour.

Expected Outcome and Plan for continuing the Work beyond ISERN session

This workshop constitutes a milestone in the development of the framework for managing scientific SE theories. We envision that the workshop will increase the rigor and relevance of the framework and ensure that its implementation will be usable. Ultimately, we hope that pursuing this initiative will provide the ISERN and ESEM community with a tool to facilitate distributed, constructive, and coherent research endeavours and produce more robust SE theories. Depending on the outcome of the workshop, we plan to run another workshop in 2025 where we present the outcome and foster its dissemination in the community under the banner of ISERN.

Expected Knowledge

Participants do not require additional knowledge beyond the basics of empirical software engineering research and the basic concepts of scientific theories.