Abstract:
Conducting Systematic Literature Reviews (SLR) has become a well-established practice in empirical software engineering. With the growing number of studies being published, it is essential to also synthesize existing research. However, this has raised the expectations on literature reviews in general, particularly as a basis for related work in empirical papers. Sometimes, reviewers ask for systematic selection of papers also for related work. Further, as conducting SLRs is a laborious task, they are often conducted by students or attempted to be automated. Finally, many SLRs focus more on rigor in the data collection than in relevant synthesis. In this session we will address the following questions: - How systematic must an SLR be, given different uses? - Who is best suited for conducting different SLR activities? - How to balance rigor and relevance in SLRs?

Session Goals:
To discuss current practices of literature reviews with respect to the three questions above. The aim is to have an impact on future conference and journal reviews, most specifically ESEM’23.

Development of the Session: (How will the session be conducted? How much interaction?)
Session attendants are expected to be familiar with SLRs practices. Hence, only a short introduction to the topic is given. Then group discussion will be held, mapping the three dimensions for SLRs: - Approach: Systematic – Semi-Systematic – Integrative (Snyder, 2019) - Conduct: Automated – Student – Senior research (van Dinter et al, 2021) - Quality criteria: Rigor – Relevance (Bandara et al, 2015)

What means for interaction will be used or required?
Group discussion around the dimensions, mixed with reporting back and discussions in the whole group.

Background and recommended reading:
H. Snyder. Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104:333–339, 2019. https://doi.org/10.1016/j.infsof.2021.106589

A. Al-Zubidy and J.C. Carver, Identification and prioritization of SLR search tool requirements: an SLR and a survey, Empir Software Eng, vol. 24, pp. 139-169, 2019. https://doi.org/10.1007/s10664-018-9626-5

R. van Dinter, B. Tekinerdogan, and C. Catal. Automation of systematic literature reviews: A systematic literature review. Information and Software Technology, 136:106589, 2021. https://doi.org/10.1016/j.infsof.2021.106589

W. Bandara, E. Furtmueller, E. Gorbacheva, S. Miskon, and J. Beekhuyzen, Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support, Communications of the Association for Information Systems: 37:8, 2015. https://doi.org/10.17705/1CAIS.03708

Expected Outcomes and Plan for Continuing the Work beyond ISERN:
If the dimensions make sense to the community, they could

a) be applied to existing SLRs to assess current status in ESE,
b) be a basis for surveys in the community, and
c) be used in reviews to assess SLRs