Abstract:

The relevance of software engineering research is often hindered by a misalignment between the research community's focus and the real-world needs of practitioners. A key factor contributing to this challenge is the incorrect formulation of research problems that resonate with industry. This thematic workshop addresses this issue by introducing a collaborative framework for research problem formulation that fosters co-creation between researchers and practitioners. The goal is to jointly formulate research problems that are not only scientifically challenging but also practically relevant. During the workshop, participants will engage in hands-on activities using na example of the framework’s application to critically assess the attributes involved in problem formulation and the dimensions essential for evaluating the relevance of these problems. Through collective reflection and discussion, the workshop aims to deepen the understanding on how to formulate relevant research problems and how to assess them before investing resources in conducting the research.

Session Goals:

• Present a framework for collaborative research problem formulation and evaluation involving researchers and practitioners.

• Discuss the main attributes to support the formulation of relevant research problems in Software Engineering. 

• Discuss the criteria to evaluate the relevance of research problems in Software Engineering.

Development of the Session: (Suggested agenda, 90 minutes in total)

1. Presentation: Overview of research problem relevance and the proposed framework (20 minutes).

2. Group Formation: Form groups and distribute materials (5 minutes).

3. Activity 1: Groups open the first envelope and discuss the problem vision statement for a specific research paper, making notes and adding post-its as needed (35 minutes).

4. Activity 2: Groups open the second envelope to discuss and evaluate the relevance of the research problem (15 minutes).

5. Activity 3: Groups open the third envelope and complete a survey on the attributes for problem formulation and the criteria for relevance assessment (15 minutes).

Background and Recommended Reading (desirable, but not needed to take part in the session):

Beecham, S., O'Leary, P., Baker, S., Richardson, I., & Noll, J. (2014). Making software engineering research relevant. Computer, 47(4), 80-83.

Carver, J. C., Dieste, O., Kraft, N. A., Lo, D., & Zimmermann, T. (2016). How practitioners perceive the relevance of esem research. In Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (pp. 1-10).

Franch, X., Mendez, D., Vogelsang, A., Heldal, R., Knauss, E., Oriol, M., ... & Zimmermann, T. (2020). How do practitioners perceive the relevance of requirements engineering research?. IEEE Transactions on Software Engineering, 48(6), 1947-1964.

Garousi, V., Borg, M., & Oivo, M. (2020). Practical relevance of software engineering research: synthesizing the community’s voice. Empirical Software Engineering, 25, 1687-1754.

Lo, D., Nagappan, N., & Zimmermann, T. (2015, August). How practitioners perceive the relevance of software engineering research. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (pp. 415-425).

*Petersen, K., Börstler, J., bin Ali, N., & Engström, E. (2024). Revisiting the construct and assessment of industrial relevance in software engineering research. In International Workshop on Methodological Issues with Empirical Studies in Software Engineering, WSESE@ICSE 2024. *

Expected Outcomes and Plan for Continuing the Work beyond ISERN:

• Better understanding on how to frame and formulate relevant research problems.

• Better understanding on how to evaluate the relevance of research problems at the beginning of a research endeavor.

• Adjustments to the framework and international multi-case studies to formulate and assess the relevance of research problems being addressed by different ISERN members.