IASESE - LLM GuidelinesESEIW 2026

This year’s International Advanced School on Empirical Software Engineering (IASESE) invites both researchers and practitioners to explore how to make LLM-based Software Engineering Studies Practically Reproducible. That is, we will showcase the effective use of the LLM Guidelines established recently by the community following an interactive and hands-on tutorial format. The tutorial will be organised as a half-day event on Wednesday and invites participants at all levels of experience.

The tutorial aims in particular to help software engineering professionals design, report, and review empirical studies involving LLMs. The session is based on the LLM Guidelines for SE as well our experience trying to replicate SE studies using LLMs (see our latest study).

We will work with concrete examples (including examples brought by the tutorial participants) to identify which guidelines to apply, and what must be reported as well as where identifying points where studies using LLMs often become difficult to reproduce. Through small-group exercises, participants will use the provided checklist and agent skill to apply the guidelines in practice.

Finally, as the guidelines are a product by the community for the community, participants will be invited to actively discuss gaps, ambiguities, and emerging use cases from their own research, helping us refine the guidelines and identify extensions and revisions that better reflect the needs of the software engineering research community.