Inclusion and Exclusion Criteria in Software Engineering Tertiary Studies: A Systematic Mapping and Emerging Framework
Context: Tertiary studies in software engineering (TS@SE) are widely used to synthesise evidence on a research topic systematically. As part of their protocol, TS@SE define inclusion and exclusion criteria (IC/EC) aimed at selecting those secondary studies (SS) to be included in the analysis.
Aims: To provide a state of the art on the definition and application of IC/EC in TS@SE, and from the results of this analysis, we outline an emerging framework, TSICEC, to be used by SE researchers.
Method: To provide the state of the art, we conducted a systematic mapping (SM) combining automatic search and snowballing over the body of SE scientific literature, which led to 50 papers after application of our own IC/EC. The extracted data was synthesised using content analysis. The results were used to define a first version of TSICEC.
Results: The SM resulted in a coding schema, and a thorough analysis of the selected papers on the basis of this coding. Our TSICEC framework includes guidelines for the definition of IC/EC in TS@SE.
Conclusion: This paper is a step forward establishing a foundation for researchers in two ways. As authors, understanding the different possibilities to define IC/EC and apply them to select SS. As readers, having an instrument to understand the methodological rigor upon which TS@SE may claim their findings.
Wed 13 OctDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:00 - 14:10 | Research MethodsEmerging Results and Vision papers / Technical Papers / Journal-first Papers at ESEM ROOM Chair(s): Tayana Conte Universidade Federal do Amazonas | ||
13:00 15mTalk | The who, what, how of software engineering research: a socio-technical framework Journal-first Papers Margaret-Anne Storey University of Victoria, Neil Ernst University of Victoria, Courtney Williams , Eirini Kalliamvakou University of Victoria | ||
13:15 15mTalk | What Evidence We would Miss If We Do not Use Grey Literature? Technical Papers Fernando Kamei Federal Institute of Alagoas (IFAL), Gustavo Pinto Federal University of Pará (UFPA) and Zup Innovation, Igor Scaliante Wiese Federal University of Technology – Paraná - UTFPR, Márcio Ribeiro Federal University of Alagoas, Brazil, Sergio Soares Informatics Center - CIn/UFPE Pre-print Media Attached | ||
13:30 10mTalk | Towards a Methodology for Participant Selection in Software Engineering Experiments. A Vision of the Future Emerging Results and Vision papers Valentina Lenarduzzi LUT University, Oscar Dieste Universidad Politécnica de Madrid, Davide Fucci Blekinge Institute of Technology, Sira Vegas Universidad Politecnica de Madrid Pre-print Media Attached | ||
13:40 10mTalk | Important Experimentation Characteristics: An Expert Survey Emerging Results and Vision papers | ||
13:50 10mTalk | Inclusion and Exclusion Criteria in Software Engineering Tertiary Studies: A Systematic Mapping and Emerging Framework Emerging Results and Vision papers Dolors Costal Universitat Politècnica de Catalunya, Carles Farré Universitat Politècnica de Catalunya, Xavier Franch Universitat Politècnica de Catalunya, Carme Quer Universitat Politècnica de Catalunya | ||
14:00 10mTalk | Towards Sustainability of Systematic Literature Reviews Emerging Results and Vision papers Vinicius Santos University of São Paulo (ICMC/USP), São Carlos - SP, Anderson Y. Iwazaki University of São Paulo (ICMC/USP), São Carlos - SP, Katia Felizardo Federal Technological University of Paraná, Érica F. Souza Federal Technological University of Paraná, Cornélio Procópio - PR, Elisa Yumi Nakagawa University of São Paulo |