ICSA 2026
Mon 22 - Fri 26 June 2026
Fri 26 Jun 2026 11:05 - 11:25 at Theatre 2 - Governance, Observability & Smells Chair(s): Sandro Speth

Architectural Decision Records (ADRs) play a central role in maintaining software architecture quality, yet many decision violations go unnoticed because projects lack both systematic documentation and automated detection mechanisms. Recent advances in Large Language Models (LLMs) open up new possibilities for automating architectural reasoning at scale. We investigated how effectively LLMs can identify decision violations in open-source systems by examining their agreement, accuracy, and inherent limitations. Our study analyzed 980 ADRs across 109 GitHub repositories using a multi-model pipeline in which one LLM primary screens potential decision violations, and three additional LLMs independently validate the reasoning. We assessed agreement, accuracy, precision, and recall, and complemented the quantitative findings with expert evaluation. The models achieved substantial agreement and strong accuracy for explicit, code-inferable decisions. Accuracy falls short for implicit or deployment-oriented decisions that depend on deployment configuration or organizational knowledge. Therefore, LLMs can meaningfully support validation of architectural decision compliance; however, they are not yet replacing human expertise for decisions not focused on code.

Fri 26 Jun

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:30 - 12:00
Governance, Observability & SmellsJournal First / Research Papers / Software Architecture Showcase / ‎​ at Theatre 2
Chair(s): Sandro Speth Technical University of Munich
10:30
20m
Research paper
Changing Nothing, Yet Changing Everything: Exploring Rug Pulls in GitHub Workflows
Research Papers
Edoardo Riggio Software Institute, Faculty of Informatics, USI Lugano, Cesare Pautasso Software Institute, Faculty of Informatics, USI Lugano
Pre-print
10:50
15m
Paper
Visualizing the Runtime Behavior of Microservices
Software Architecture Showcase
Supun Wijayakoon Eindhoven University of Technology, Filip Zamfirov , Jacob Krüger Eindhoven University of Technology, Michel Chaudron Eindhoven University of Technology, The Netherlands, Tomas Molina
Pre-print
11:05
20m
Research paper
Evaluating Large Language Models for Detecting Architectural Design Rules ViolationsDistinguished Paper Award
Research Papers
Ruoyu Su M3S, University of Oulu, Alexander Bakhtin University of Oulu, Noman Ahmad University of Oulu, Matteo Esposito University of Oulu, Valentina Lenarduzzi University of Southern Denmark, Davide Taibi University of Southern Denmark and University of Oulu
Pre-print
11:25
20m
Research paper
TwinArch: A Digital Twin Reference ArchitectureJournal First
Journal First
Alessandra Somma University of Naples Federico II, Domenico Amalfitano University of Naples Federico II, Alessandra De Benedictis , Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy