ICSE 2024
Fri 12 - Sun 21 April 2024 Lisbon, Portugal

Software engineering (SE) is full of abstract concepts that are crucial for both researchers and practitioners, such as programming experience, team productivity, code comprehension, and system security. Secondary studies aimed at summarizing research on the influences and consequences of such concepts would therefore be of great value.

However, the inability to measure abstract concepts directly poses a challenge for secondary studies: primary studies in SE can operationalize such concepts in many ways. Standardized measurement instruments are rarely available, and even if they are, many researchers do not use them or do not even provide a definition for the studied concept. SE researchers conducting secondary studies therefore have to decide a) which primary studies intended to measure the same construct, and b) how to compare and aggregate vastly different measurements for the same construct.

In this experience report, we discuss the challenge of study selection in SE secondary research on latent variables. We report on two instances where we found it particularly challenging to decide which primary studies should be included for comparison and synthesis, so as not to end up comparing apples with oranges. Our report aims to spark a conversation about developing strategies to address this issue systematically and pave the way for more efficient and rigorous secondary studies in software engineering.

Tue 16 Apr

Displayed time zone: Lisbon change

11:00 - 12:30
Session 2 - Quality of evidence generated & Secondary StudiesWSESE at Eugénio de Andrade
Chair(s): Andreas Jedlitschka Fraunhofer IESE
11:00
12m
Talk
Evidence Tetris in the Pixelated World of Validity Threats
WSESE
Marvin Wyrich Saarland University, Sven Apel Saarland University
Pre-print
11:12
12m
Talk
Revisiting the construct and assessment of industrial relevance in software engineering research
WSESE
Kai Petersen Blekinge Institute of Technology/Ericsson AB, J�rgen B�rstler Blekinge Institute of Technology, Nauman Bin Ali Blekinge Institute of Technology, Emelie Engstrom Lund University
11:24
15m
Talk
Grounded Theory in Software Engineering: Challenges and Lessons Learned from the Trenches
WSESE
Pilar Rodriguez Universidad Politécnica de Madrid
11:39
18m
Talk
A Second Look at the Impact of Passive Voice Requirements on Domain Modeling: Bayesian Reanalysis of an Experiment
WSESE
Julian Frattini Blekinge Institute of Technology, Davide Fucci Blekinge Institute of Technology, Richard Torkar Chalmers and the University of Gothenburg, Daniel Mendez Blekinge Institute of Technology and fortiss
DOI Pre-print
11:57
18m
Talk
Emerging Results on Automated Support for Searching and Selecting Evidence for Systematic Literature Review Updates
WSESE
Bianca Minetto Napoleão Université du Québec à Chicoutimi, Ritika Sarkar Université du Québec à Chicoutimi, Sylvain Hallé Université du Québec à Chicoutimi, Fabio Petrillo École de technologie supérieure (ÉTS), Montréal -- Université du Québec, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio)
12:15
15m
Talk
Apples, Oranges, and Software Engineering: Study Selection Challenges for Secondary Research on Latent Variables
WSESE
Marvin Wyrich Saarland University, Marvin Muñoz Barón University of Stuttgart, Justus Bogner Vrije Universiteit Amsterdam
Pre-print