Towards a Methodology for Participant Selection in Software Engineering Experiments. A Vision of the Future
Background. Software Engineering (SE) researchers extensively perform experiments with human subjects. Well-defined samples are required to ensure external validity. Samples are selected purposely or by convenience, limiting the generalizability of results.
Objective. We aim to depict the current status of participants selection in empirical SE, identifying the main threats and how they are mitigated. We draft a robust approach to participants’ selection.
Method. We reviewed existing participants’ selection guidelines in SE, and performed a preliminary literature review to find out how participants’ selection is conducted in SE in practice.
Results. We outline a new selection methodology, by 1) defining the characteristics of the desired population, 2) locating possible sources of sampling available for researchers, and 3) identifying and reducing the “distance” between the selected sample and its corresponding population.
Conclusion. We propose a roadmap to develop and empirically validate the selection methodology.
Wed 13 OctDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:00 - 14:10 | Research MethodsEmerging Results and Vision papers / Technical Papers / Journal-first Papers at ESEM ROOM Chair(s): Tayana Conte Universidade Federal do Amazonas | ||
13:00 15mTalk | The who, what, how of software engineering research: a socio-technical framework Journal-first Papers Margaret-Anne Storey University of Victoria, Neil Ernst University of Victoria, Courtney Williams , Eirini Kalliamvakou University of Victoria | ||
13:15 15mTalk | What Evidence We would Miss If We Do not Use Grey Literature? Technical Papers Fernando Kamei Federal Institute of Alagoas (IFAL), Gustavo Pinto Federal University of Pará (UFPA) and Zup Innovation, Igor Scaliante Wiese Federal University of Technology – Paraná - UTFPR, Márcio Ribeiro Federal University of Alagoas, Brazil, Sergio Soares Informatics Center - CIn/UFPE Pre-print Media Attached | ||
13:30 10mTalk | Towards a Methodology for Participant Selection in Software Engineering Experiments. A Vision of the Future Emerging Results and Vision papers Valentina Lenarduzzi LUT University, Oscar Dieste Universidad Politécnica de Madrid, Davide Fucci Blekinge Institute of Technology, Sira Vegas Universidad Politecnica de Madrid Pre-print Media Attached | ||
13:40 10mTalk | Important Experimentation Characteristics: An Expert Survey Emerging Results and Vision papers | ||
13:50 10mTalk | Inclusion and Exclusion Criteria in Software Engineering Tertiary Studies: A Systematic Mapping and Emerging Framework Emerging Results and Vision papers Dolors Costal Universitat Politècnica de Catalunya, Carles Farré Universitat Politècnica de Catalunya, Xavier Franch Universitat Politècnica de Catalunya, Carme Quer Universitat Politècnica de Catalunya | ||
14:00 10mTalk | Towards Sustainability of Systematic Literature Reviews Emerging Results and Vision papers Vinicius Santos University of São Paulo (ICMC/USP), São Carlos - SP, Anderson Y. Iwazaki University of São Paulo (ICMC/USP), São Carlos - SP, Katia Felizardo Federal Technological University of Paraná, Érica F. Souza Federal Technological University of Paraná, Cornélio Procópio - PR, Elisa Yumi Nakagawa University of São Paulo |