What Evidence We would Miss If We Do not Use Grey Literature?
Context: Multivocal Literature Reviews (MLR) search for evidence in both Traditional Literature (TL) and Grey Literature (GL). Despite the growing interest in MLR-based studies, the literature assessing how GL has contributed to MLR studies is still scarce.
Objective: This research aims to assess how the use of GL contributed to MLR studies. By contributing, we mean, understanding to what extent GL is providing evidence that is indeed used by an MLR to answer its research question.
Method: We start by conducting a tertiary study to identify MLR studies published between 2017 and 2019, selecting nine of them. We then identified the GL used in these studies and assessed to what extent the GLs are providing evidence that help these studies to answer their research questions.
Results: Our analysis identified that 1) GL provided evidence not found in TL, 2) most of the GL sources were used to provide recommendations to solve problems, explain a topic, and classify the findings, and 3) 19 different GL types were used in the studies; these GLs were mainly produced by SE practitioners (including blog posts, slides presentations, or project descriptions).
Conclusions: We evidence how GL contributed to MLR studies. We observed that if these GLs were not included in the MLR, several findings would have been omitted or weakened. We also described the challenges involved when conducting this investigation, along with potential ways to deal with them, which may help future SE researchers.
Wed 13 OctDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
| 13:00 - 14:10 | Research MethodsEmerging Results and Vision papers / Technical Papers / Journal-first Papers at ESEM ROOM  Chair(s): Tayana Conte Universidade Federal do Amazonas | ||
| 13:0015m Talk | The who, what, how of software engineering research: a socio-technical framework Journal-first Papers Margaret-Anne Storey University of Victoria, Neil Ernst University of Victoria, Courtney Williams , Eirini Kalliamvakou University of Victoria | ||
| 13:1515m Talk | What Evidence We would Miss If We Do not Use Grey Literature? Technical Papers Fernando Kamei Federal Institute of Alagoas (IFAL), Gustavo Pinto Federal University of Pará (UFPA) and Zup Innovation, Igor Scaliante Wiese Federal University of Technology – Paraná - UTFPR, Márcio Ribeiro Federal University of Alagoas, Brazil, Sérgio Soares Informatics Center - CIn/UFPEPre-print Media Attached | ||
| 13:3010m Talk | Towards a Methodology for Participant Selection in Software Engineering Experiments. A Vision of the Future Emerging Results and Vision papers Valentina Lenarduzzi LUT University, Oscar Dieste Universidad Politécnica de Madrid, Davide Fucci Blekinge Institute of Technology, Sira Vegas Universidad Politecnica de MadridPre-print Media Attached | ||
| 13:4010m Talk | Important Experimentation Characteristics: An Expert Survey Emerging Results and Vision papers | ||
| 13:5010m Talk | Inclusion and Exclusion Criteria in Software Engineering Tertiary Studies: A Systematic Mapping and Emerging Framework Emerging Results and Vision papers Dolors Costal Universitat Politècnica de Catalunya, Carles Farré Universitat Politècnica de Catalunya, Xavier Franch Universitat Politècnica de Catalunya, Carme Quer Universitat Politècnica de Catalunya | ||
| 14:0010m Talk | Towards Sustainability of Systematic Literature Reviews Emerging Results and Vision papers Vinicius Santos University of São Paulo (ICMC/USP), São Carlos - SP, Anderson Y. Iwazaki University of São Paulo (ICMC/USP), São Carlos - SP, Katia Felizardo Federal Technological University of Paraná, Érica F. Souza Federal Technological University of Paraná, Cornélio Procópio - PR, Elisa Yumi Nakagawa University of São Paulo | ||




