Qualitative Surveys in Software Engineering Research: Definition, Critical Review, and Guidelines
This program is tentative and subject to change.
Qualitative surveys are emerging as a popular research method in software engineering (SE), particularly as many aspects of the field are increasingly socio-technical and thus concerned with the subtle, social, and often ambiguous issues that are not amenable to a simple quantitative survey. While many argue that qualitative surveys play a vital role amongst the diverse range of methods employed in SE there are a number of shortcomings that inhibits its use and value. First there is a lack of clarity as to what defines a qualitative survey and what features differentiate it from other methods. There is an absence of a clear set of principles and guidelines for its execution, and what does exist is very inconsistent and sometimes contradictory. These issues undermine the perceived reliability and rigour of this method. Researchers are unsure about how to ensure reliability and rigour when designing qualitative surveys and reviewers are unsure how these should be evaluated. In this paper, we present a systematic mapping study to identify how qualitative surveys have been employed in SE research to date. This paper proposes a set of principles, based on a multidisciplinary review of qualitative surveys and capturing some of the commonalities of the diffuse approaches found. These principles can be used by researchers when choosing whether to do a qualitative survey or not. They can then be used to design their study. The principles can also be used by editors and reviewers to judge the quality and rigour of qualitative surveys. It is hoped that this will result in more widespread use of the method and also more effective and evidence-based reviews of studies that use these methods in the future.
This program is tentative and subject to change.
Fri 2 MayDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 17:30 | |||
16:00 15mTalk | Full Line Code Completion: Bringing AI to Desktop SE In Practice (SEIP) Anton Semenkin JetBrains, Vitaliy Bibaev JetBrains, Yaroslav Sokolov JetBrains, Kirill Krylov JetBrains, Alexey Kalina JetBrains, Anna Khannanova JetBrains, Danila Savenkov JetBrains, Darya Rovdo JetBrains, Igor Davidenko JetBrains, Kirill Karnaukhov JetBrains, Maxim Vakhrushev JetBrains, Mikhail Kostyukov JetBrains, Mikhail Podvitskii JetBrains, Petr Surkov JetBrains, Yaroslav Golubev JetBrains Research, Nikita Povarov JetBrains, Timofey Bryksin JetBrains Research Pre-print | ||
16:15 15mTalk | Automated Accessibility Analysis of Dynamic Content Changes on Mobile Apps Research Track Forough Mehralian University of California at Irvine, Ziyao He University of California, Irvine, Sam Malek University of California at Irvine | ||
16:30 15mTalk | Qualitative Surveys in Software Engineering Research: Definition, Critical Review, and Guidelines Journal-first Papers Jorge Melegati Free University of Bozen-Bolzano, Kieran Conboy University of Galway, Daniel Graziotin University of Hohenheim | ||
16:45 15mTalk | VulNet: Towards improving vulnerability management in the Maven ecosystem Journal-first Papers Zeyang Ma Concordia University, Shouvick Mondal IIT Gandhinagar, Tse-Hsun (Peter) Chen Concordia University, Haoxiang Zhang Centre for Software Excellence at Huawei Canada, Ahmed E. Hassan Queen’s University, Zeyang Ma Concordia University | ||
17:00 15mTalk | Energy-Aware Software Testing New Ideas and Emerging Results (NIER) Roberto Verdecchia University of Florence, Emilio Cruciani University of Salzburg, Antonia Bertolino National Research Council, Italy, Breno Miranda Centro de Informática at Universidade Federal de Pernambuco Pre-print | ||
17:15 7mTalk | SusDevOps: Promoting Sustainability to a First Principle in Software Delivery New Ideas and Emerging Results (NIER) Istvan David McMaster University / McMaster Centre for Software Certification (McSCert) |