ICSE 2026
Sun 12 - Sat 18 April 2026 Rio de Janeiro, Brazil

Context: The empirical software engineering (ESE) community has contributed to improving experimentation over the years. However, there is still a lack of rigor in describing controlled experiments, hindering reproducibility and transparency. Registered Reports (RR) have been discussed in the ESE community to address these issues. A RR registers a study’s hypotheses, methods, and/or analyses before execution, involving peer review and potential acceptance before data collection. This helps mitigate problematic practices such as p-hacking, publication bias, and inappropriate post hoc analysis. Objective: This paper presents initial results toward establishing an RR template for Software Engineering controlled experiments using the Open Science Framework (OSF). Method: We analyzed templates of selected OSF RR type in light of documentation guidelines for controlled experiments. Results: The observed lack of rigor motivated our investigation of OSF-based RR types. Our analysis showed that, although one of the RR types aligned with many of the documentation suggestions contained in the guidelines, none of them covered the guidelines comprehensively. The study also highlights limitations in OSF RR template customization. Conclusion: Despite progress in ESE, planning and documenting experiments still lack rigor, compromising reproducibility. Adopting OSF-based RRs is proposed. However, no currently available RR type fully satisfies the guidelines. Establishing RR-specific guidelines for SE is deemed essential.

Mon 13 Apr

Displayed time zone: Brasilia, Distrito Federal, Brazil change

09:00 - 10:30
Opening, Keynote, Controlled ExperimentsWSESE at Oceania III
Chair(s): Silvia Abrahão Universitat Politècnica de València, Patricia Lago Vrije Universiteit Amsterdam, Sira Vegas Universidad Politecnica de Madrid
09:00
15m
Day opening
Welcome by the Organizers
WSESE
Patricia Lago Vrije Universiteit Amsterdam, Silvia Abrahão Universitat Politècnica de València, Sira Vegas Universidad Politecnica de Madrid
09:15
50m
Keynote
We don’t know how to best use LLMs for qualitative analysis… But maybe we know how not to
WSESE
Carolyn Seaman University of Maryland Baltimore County
10:05
18m
Full-paper
Towards an OSF-based Registered Report Template for Software Engineering Controlled ExperimentsTechnical Paper
WSESE
Ana B. M. Bett State University of Maringá, Thais S. Nepomuceno State University of Maringá, Edson OliveiraJr State University of Maringá, Maria Teresa Baldassarre Department of Computer Science, University of Bari , Valdemar Graciano Neto Federal University of Goiás, Marcos Kalinowski Pontifical Catholic University of Rio de Janeiro (PUC-Rio)
Pre-print
10:23
7m
Panel
Discussion of Controlled Experiments
WSESE