Communicating Study Design Trade-offs in Software Engineering
Reflecting on the limitations of a study is a crucial part of the research process. In software engineering studies, this reflection is typically conveyed through discussions of study limitations or threats to validity. In current practice, such discussions seldom provide sufficient insight to understand the rationale for decisions taken before and during the study, and their implications. We revisit the practice of discussing study limitations and threats to validity and identify its weaknesses. We propose to refocus this practice of self-reflection to a discussion centered on the notion of trade-offs. We argue that documenting trade-offs allows researchers to clarify how the benefits of their study design decisions outweigh the costs of possible alternatives. We present guidelines for reporting trade-offs in a way that promotes a fair and dispassionate assessment of researchers’ work.
Thu 4 SepDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
11:00 - 12:30 | Education and Research PracticeJournal-First / Research Papers / RE@Next! Papers at Salon de Grados Chair(s): Paola Spoletini Kennesaw State University | ||
11:00 30mPaper | Students’ Perception of LLM Use in Requirements Engineering Education: An Empirical Study Across Two Universities Research Papers Sharon Clarissa Guardado Medina University of Oulu, Risha Parveen , Zheying Zhang Tampere University, Maruf Rayhan Tampere University, Nirnaya Tripathi University of Oulu | ||
11:30 20mPaper | Leveraging LLMs for Requirements Engineering Education: How to Approach? RE@Next! Papers Saurabh Tiwari Dhirubhai Ambani University, formerly DA-IICT, Gandhinagar, Santosh Singh Rathore ABV-Indian Institute of Information Technology and Management Gwalior | ||
11:50 20mPaper | Rethinking RE Topic Mapping: Toward an Extensible Framework for Curriculum–Industry Comparison RE@Next! Papers Anthea Moravánszky University of Szeged, Hungary; University of Applied Sciences of the Grisons, Switzerland, Ingo Barkow | ||
12:10 20mPaper | Communicating Study Design Trade-offs in Software Engineering Journal-First Martin P. Robillard McGill University, Deeksha M. Arya McGill University, Neil Ernst University of Victoria, Jin L.C. Guo McGill University, Maxime Lamothe Polytechnique Montreal, Montreal, Canada, Mathieu Nassif McGill University, Nicole Novielli University of Bari, Alexander Serebrenik Eindhoven University of Technology, Igor Steinmacher NAU RESHAPE LAB, Klaas-Jan Stol Lero; University College Cork; SINTEF Digital Link to publication DOI | ||