How to Elicit Explainability Requirements? A Comparison of Interviews, Focus Groups, and Surveys
As software systems grow increasingly complex, explainability has become a crucial non-functional requirement for transparency, user trust, and regulatory compliance. Eliciting explainability requirements is challenging, as different methods capture varying levels of detail and structure. This study examines the efficiency and effectiveness of three commonly used elicitation methods—focus groups, interviews, and online surveys—while also assessing the role of taxonomy usage in structuring and improving the elicitation process. We conducted a case study at a large German IT consulting company, utilizing a web-based personnel management software. A total of two focus groups, 18 interviews, and an online survey with 188 participants were analyzed. The results show that interviews were the most efficient, capturing the highest number of distinct needs per participant per time spent. Surveys collected the most explanation needs overall but had high redundancy. Delayed taxonomy introduction resulted in a greater number and diversity of needs, suggesting that a two-phase approach is beneficial. Based on our findings, we recommend a hybrid approach combining surveys and interviews to balance efficiency and coverage. Future research should explore how automation can support elicitation and how taxonomies can be better integrated into different methods.
Wed 3 SepDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
16:00 - 17:40 | Explainability and Ethics IIJournal-First / Research Papers / RE@Next! Papers at Salon de Actos Chair(s): Chetan Arora Monash University | ||
16:00 30mPaper | How to Elicit Explainability Requirements? A Comparison of Interviews, Focus Groups, and Surveys Research Papers Martin Obaidi Leibniz Universität Hannover, Jakob Droste Leibniz Universität Hannover, Hannah Deters Leibniz University Hannover, Marc Herrmann Leibniz University Hannover, Jil Klünder University of Applied Sciences | FHDW Hannover, Kurt Schneider Leibniz Universität Hannover, Software Engineering Group, Raymond Ochsner Leibniz Universität Hannover Pre-print | ||
16:30 30mPaper | Design Thinking In Requirements Engineering: Understanding The Role Of Internal And External Empathy Research Papers Ezequiel Kahan Universidad Nacional de Tres de Febrero, Marcela Fabiana Genero Bocco University of Castilla-La Mancha, Beatriz Bernárdez University of Seville, Alejandro Oliveros Universidad Nacional de Tres de Febrero | ||
17:00 20mPaper | Explainability Across the Spectrum: Modeling Stakeholder Goals Based on AI Complexity Levels RE@Next! Papers Antoni Mestre Gascón Universitat Politècnica de València, Manoli Albert Universitat Politecnica de Valencia, Miriam Gil Universidad de Valencia, Vicente Pelechano Universitat Politècnica de València | ||
17:20 20mPaper | ExplanaSC: A Framework for Determining Information Requirements for Explainable Blockchain Smart Contracts Journal-First |