Research TrackREFSQ 2024

Requirements Engineering (RE) is a critical factor in developing high-quality and successful software, systems, and services. The REFSQ working conference series is an established international forum for discussing current and state-of-the-art RE practices, celebrating its 30th edition.

Accepted Papers

In response to the Call for Papers, we received submissions, out of which (after initial screening) 53 were peer-reviewed by at least three program committee member and extensively discussed among the reviewers. Seven papers for which no consensus had then been accepted on the condition that certain improvements be made (those underwent an additional check by a PC member before final acceptance).

Overall, 22 papers were finally accepted for publication, leading to an acceptance rate of 37% (22/59).

List of accepted papers

	Title
	A New Usability Inspection Method: Experience-based AnalysisTechnical design Research Track P: Anu Piirisild, A: Ana Perandrés Gómez, A: Kuldar Taveter, D: Preethu Rose Anish File Attached
	Assessing the Understandability of Attack-Defense Trees for Modelling Security Requirements: an Experimental InvestigationScientific evaluation Research Track P: Giovanna Broccia, A: Maurice ter Beek, A: Alberto Lluch Lafuente, A: Paola Spoletini, A: Alessio Ferrari, D: Guntur Budi Herwanto File Attached
	A tertiary study on AI for Requirements EngineeringScientific evaluation Research Track P: Ali Mehraj, A: Zheying Zhang, A: Kari Systa, D: Laura Semini
	Behavior-Driven Specification in Practice: An Experience ReportExperience Report Research Track A: Joel Allred, A: Simon Fraser, P: Alessandro Pezzoni, D: Paola Spoletini
	Candidate Solutions for Defining Explainability Requirements of AI Systems CANCELLED Research Track P: Nagadivya Balasubramaniam, A: Marjo Kauppinen, A: Hong-Linh Truong, A: Sari Kujala, D: Rebekka Wohlrab
	Channeling the Voice of the Crowd: Applying Structured Queries in User Feedback CollectionTechnical design Research Track A: Leon Radeck, P: Barbara Paech, D: Quim Motger
	Designing NLP-based solutions for requirements variability management: experiences from a design science study at VismaExperience Report Research Track P: Parisa Elahidoost, A: Michael Unterkalmsteiner, A: Davide Fucci, A: Peter Liljenberg, A: Jannik Fischbach, D: Rim Zrelli
	Exploring LLMs' ability to detect variability in requirementsResearch Preview Research Track A: Alessandro Fantechi, A: Stefania Gnesi, P: Laura Semini, D: Jill Tamanini
	Exploring the Automatic Classification of Usage Information in FeedbackScientific evaluation Research Track P: Michael Anders, A: Barbara Paech, A: Lukas Bockstaller, D: Alessandro Pezzoni
	Governance-focused Classification of Security and Privacy Requirements from Obligations in Software Engineering ContractsTechnical design Research Track P: Preethu Rose Anish, A: Aparna Verma, A: Sivanthy Venkatesan, A: Logamurugan V, A: Smita Ghaisas, D: Anu Piirisild
	How Explainable is your System? Towards a Quality Model for ExplainabilityTechnical design Research Track P: Hannah Deters, A: Jakob Droste, A: Martin Obaidi, A: Kurt Schneider, D: Julian Frattini
	Identifying relevant Factors of Requirements Quality: an industrial Case StudyScientific evaluation Research Track P: Julian Frattini, D: Hannah Deters Pre-print
	Learning to Rank Privacy Design Patterns: A Semantic Approach to Meeting Privacy RequirementsTechnical design Research Track P: Guntur Budi Herwanto, A: Gerald Quirchmayr, A: A Min Tjoa, D: Giovanna Broccia
	Natural2CTL: A Dataset for Natural Language Requirements and their CTL Formal EquivalentsResearch Preview Research Track P: Rim Zrelli, A: Henrique Amaral Misson, A: Maroua Ben Attia, A: Felipe Gohring de Magalhães, A: Abdo Shabah, A: Gabriela Nicolescu, D: Parisa Elahidoost
	Operationalizing Machine Learning Using Requirements-Grounded MLOpsTechnical design Research Track A: Milos Bastajic, A: Jonatan Boman Karinen, P: Jennifer Horkoff, D: Meira Levy
	Opportunities and Limitations of AI in Human-Centered Design - A Research PreviewResearch Preview Research Track A: Anne Hess, A: Thomas Immich, P: Jill Tamanini, A: Mario Biedenbach, A: Matthias Koch, D: Ali Mehraj
	Requirements Engineering for No-Code Development (RE4NCD): A Case Study of Rapid Application Development during War Research Preview Research Track P: Meira Levy, A: Irit Hadar, D: Eran Sadovski
	Requirements Information in Backlog Items: Content AnalysisScientific evaluation Research Track P: Ashley van Can, A: Fabiano Dalpiaz, D: Eduard C. Groen
	Sustainable Software or Sustainable Business Models Research Track K: Lorenz Hilty File Attached
	The Return of Formal Requirements Engineering in the Era of Large Language ModelsVision Paper Research Track P: Paola Spoletini, A: Alessio Ferrari, D: Michael Anders
	Towards a Comprehensive Ontology for Requirements Engineering for AI-powered SystemsResearch Preview Research Track P: Eran Sadovski, A: Itzhak Aviv, A: Irit Hadar, D: Jennifer Horkoff
	Unveiling Competition Dynamics in Mobile App Markets through User ReviewsTechnical design Research Track P: Quim Motger, A: Xavier Franch, A: Vincenzo Gervasi, A: Jordi Marco, D: Barbara Paech Pre-print
	What Impact do my Preferences Have? A Framework for Explanation-Based Elicitation of Quality Objectives for Robotic Mission PlanningTechnical design Research Track P: Rebekka Wohlrab, A: Michael Vierhauser, A: Erik Nilsson, D: Nagadivya Balasubramaniam

Call for Papers

We invite submissions along the following categories

• Technical design papers (15 pages incl. references) describe the design of new artifacts, i.e., novel solutions for problems relevant to practice and/or significant and theoretically sound improvements of existing solutions. A preliminary validation of the artifacts is also expected.

• Scientific evaluation papers (15 pages incl. references) investigate existing real-world problems, evaluate existing artifacts implemented in real-world settings, or validate newly designed artifacts, e.g., by means such as case studies, action research, quasi-controlled experiments, simulations, surveys, or secondary studies if they clearly synthesize the state of reported evidence in literature (via systematic literature reviews or mapping studies). Please refer also to the Empirical Standards for guidelines and review criteria for each research method: https://github.com/acmsigsoft/EmpiricalStandards

• Experience report papers (12 pages incl. references) describe retrospective reports on experiences in applying RE techniques in practice, or addressing RE problems in real-world contexts. These papers focus on reporting the experience and give special attention to practical insights, lessons learned, and/or key takeaways and recommendations to the community. Experience reports may also include studies in which the authors interview practitioners about the application of specific RE techniques or about RE problems in practice.

• Vision papers (8 pages incl. references) state where research in the field should be heading.

• Research previews (8 pages incl. references) describe well-defined research ideas at an early stage of investigation which may not be fully developed.

Each type of paper has its own review criteria, which are listed here.

Finally, we cordially invite authors to disclose their research artifacts following our open science guidelines. Authors who wish to disclose their artifacts can find further guidance and support under the open science initiative.

Submission, Reviewing and Publication

Contributions must be submitted to easychair

Each submission in the scope of REFSQ will undergo a single-blind review process that will involve at least three members of the program committee.

The REFSQ 2024 proceedings will be published in Springer’s LNCS series.

A selection of best papers will be invited to submit extended versions for tentative publication in a Special Section of the journal of Information and Software Technology published by Elsevier https://www.journals.elsevier.com/information-and-software-technology

Formatting

All submissions must be formatted according to the Springer LNCS/LNBIP conference proceedings template (for LaTeX and Word), available here. As per the guidelines, please remember to include keywords after your abstract.

Furthermore, to facilitate accurate bidding and a better understanding of the papers, each paper submitted to REFSQ 2024 is required to have an abstract structured with exactly 4 paragraphs with the following content:

• Context and motivation: situate and motivate your research.

• Question/problem: formulate the specific question/problem addressed by the paper.

• Principal ideas/results: summarize the ideas and results described in your paper. State, where appropriate, your research approach and methodology.

• Contribution: state the main contribution of your paper, by highlighting its added value (e.g., to theory, to practice). Also, state the limitations of your results.

Open Science Policy

REFSQ encourages sharing data, code, and other artifacts, and this year it features a dedicated Open Science track. No artifact submissions are required, but the authors are encouraged to provide an availability statement and follow the guidelines reported on the web page of the Open Science Track

REFSQ-RE@Next! Transfer Model

This year, we are experimenting with a new transfer model between the REFSQ research paper track and RE@Next! track of the RE conference to bring the conferences closer together. REFSQ submissions which are not yet deemed mature enough to be accepted for presentation at REFSQ but are considered promising as RE@Next! contributions, will have the opportunity to participate in the transfer model.

In close collaboration between the PC co-chairs of the REFSQ research track and the RE@Next! track, this model will invite authors of selected submissions to revise their manuscript and submit it to RE@Next! with a rebuttal of changes suggested by the chairs. Those submissions will then not undergo a regular review process, but will be evaluated by the chairs of both tracks.

Each paper category has its own review criteria. We invite authors and reviewers to check the criteria and consider their order of relevance.

Technical design papers (15 pages incl. references) describe the design of new artifacts, i.e., novel solutions for requirements-related problems or significant improvements of existing solutions. A preliminary evaluation of the artifacts is also expected.

Review Criteria (in order of relevance):

• Soundness: has the novel solution been developed according to recognised research methods? Is the preliminary evaluation of the solution sound? Did the authors clearly state the research questions? Are the conclusions of the preliminary evaluation logically derived from the data? Did the authors discuss the limitations of the proposal?
• Novelty: to what extent is the proposed solution novel with respect to the state-of-the-art? To what extent is related literature considered? To what extent did the authors clarify their contribution? [NOTE: The potential lack of novelty is NOT an argument for rejection, but we expect authors to clearly convey the novelty of their contribution in light of the existing body of knowledge (including and especially when submitting replication studies)]
• Potential Impact/Relevance: is the potential impact on research and practice clearly stated? Is the potential impact convincing? Has the proposed solution been preliminarily evaluated in a representative setting?
• Verifiability: did the authors share their software? Did the authors share their data? Did the authors share their material? Did the authors provide guidelines on how to reuse their artfiacts and replicate their results? [NOTE: sharing data and software is NOT mandatory, but papers that make an effort in this direction should be adequately rewarded]
• Presentation: is the paper clearly presented? To what extent can the content of the paper be understood by the general RE public? If highly technical content is presented, did the authors make an effort to also summarise their proposal in an intuitive way?

Scientific evaluation papers (15 pages incl. references) investigate existing real-world problems, evaluate existing real-world implemented artifacts, or validate newly designed artifacts, e.g., by means of case studies, experiments, simulation, surveys, systematic literature reviews, mapping studies, or action research. Check the Empirical Standards for guidelines and review criteria for each research stretegy.

Review Criteria (in order of relevance):

• Soundness: has the novel solution been developed according to recognised research methods? Is the research method justified? Is the research method adequate for the problem at hand? Did the authors clearly state the research questions, data collection, and analysis? Are the conclusions of the evaluation logically derived from the data? Did the authors discuss the threats to validity?
• Potential Impact: is the potential impact on research and practice clearly stated? Is the potential impact convincing? Was the study carried out in a representative setting?
• Verifiability: did the authors share their software? Did the authors share their data? Did the authors provide guidelines on how to reuse their artfiacts and replicate their results? [NOTE: sharing data and software is NOT mandatory, but papers that make an effort in this direction should be adequately rewarded]
• Novelty: to what extent is the proposed solution novel with respect to the state-of-the-art? To what extent is related literature considered? To what extent did the authors clarify their contribution? [NOTE: The potential lack of novelty is NOT an argument for rejection, but we expect authors to clearly convey the novelty of their contribution in light of the existing body of knowledge (including and especially when submitting replication studies)]
• Presentation: is the paper clearly presented? To what extent can the content of the paper be understood by the general RE public? If highly technical content is presented, did the authors make an effort to also summarise their study in an intuitive way?

Experience report papers (12 pages incl. references) describe retrospective reports on experiences in applying RE techniques in practice, or addressing RE problems in real-world contexts. These papers focus on reporting the experience in a narrative form, and give prominence to the lessons learned by the authors and/or by the participants.

Review Criteria (in order of relevance):

• Relevance of the Application: is the application context in which the experience is carried out interesting for the RE public? Is the application context sufficiently representative? To what extent is the paper reporting a real-world experience involving practitioners? Is the experience credible?
• Relevance of Lessons Learned: are the lessons learned sufficiently insightful? Did the authors report convincing evidence, also anecdotal, to justify the lessons learned?
• Potential for Discussion: will the presentation of the paper raise discussion at the REFSQ conference? To what extent can REFSQ participants take inspiration to develop novel solutions based on the reported experience? To what extent can REFSQ participants take inspiration to perform sound empirical evaluations based on the reported experience?
• Novelty: is the context of the study in line with the current RE practice? Does the study report on a contemporary problem that RE practitioners and researchers typically face?

• Presentation: is the application context clearly presented? Are the lessons learned clearly described? To what extent can the content of the paper be understood by the general RE public?

Vision papers (8 pages incl. references) state where research in the field should be heading.

Review Criteria (in order of relevance):

• Potential Impact: will the vision impact the future research and practice in RE? Is a roadmap discussed? Is the vision sufficiently broad to affect different subfields of RE? Do the authors discuss both short-term and long-term impacts of their vision?
• Potential for Discussion: will the presentation of the vision raise the interest of the REFSQ audience? Will the vision raise discussion? Can the vision raise controversial opinions in the audience?
• Novelty: is the vision sufficiently novel with respect to existing reflections within the REFSQ community? Do the authors clarify the novelty of their vision?
• Soundness of Arguments: is the vision supported by logical arguments? Are the implications convincing?
• Presentation: is the vision presented in a compelling way? Is the vision presented in a way that can elicit reflections in the RE community?

Research previews (8 pages incl. references) describe well-defined research ideas at an early stage of investigation which may not be fully developed.

Review Criteria (in order of relevance):

• Novelty: did the research preview make you say “I heard it first at REFSQ!”? Is the idea sufficiently novel with respect to the state-of-the-art? Do the authors discuss related work and the contribution of their study?
• Soundness of the Research Plan: do the authors present a convincing research plan? Did the authors discuss the limitations and risks of their plan? Is the plan referring to sound research methods? Do the authors clarify their research questions, planned data collection, and data analysis? Did the authors perform a convincing proof-of-concept or preliminary research step?
• Potential for Discussion: will the presentation of the preview raise the interest of the REFSQ audience? Will the preview raise discussion? Will the audience be able to provide useful feedback to the authors, given the typical background of the REFSQ audience? Can the preview raise controversial opinions in the audience?
• Presentation: is the paper clearly presented? To what extent can the content of the paper be understood by the general RE public?