ECSA 2025
Mon 15 - Fri 19 September 2025 Limassol, Cyprus
Fri 19 Sep 2025 12:00 - 12:15 at Phoenix - Session 7 - LLMs in Software Architecture (II) Chair(s): Salah Sadou

This works explores the feasibility and challenges of using a Large Language Model (LLM) to automatically assess the quality of software architecture diagrams. Our approach is based on a structured prompt that guides the LLM to evaluate architecture diagrams and their accompanying descriptions according to five core quality criteria: clarity, consistency, completeness, accuracy, and level of detail. Preliminary experimental results using OpenAI’s ChatGPT-4o in four open-source projects suggest that LLMs can provide valuable feedback and detect diagrammatic inconsistencies, often in alignment with human expert evaluations. However, the LLM also struggled with context-specific design choices, sometimes misjudging deliberate omissions or the appropriate level of detail, indicating that human oversight remains indispensable. To guide researchers and practitioners, we further recommend practical guidelines for data preparation, prompt construction, and result interpretation, aiming to maximize the reliability and utility of LLM-based architectural evaluations.

Fri 19 Sep

Displayed time zone: Athens change

11:00 - 12:30
Session 7 - LLMs in Software Architecture (II)Research Papers / Industry Program at Phoenix
Chair(s): Salah Sadou IRISA, University of South Brittany
11:00
30m
Full-paper
Automated Software Architecture Design Recovery from Source Code Using LLMsResearch Track Paper
Research Papers
Tiziano Santilli Gran Sasso Science Institute (GSSI), Marco De Luca University of Naples Federico II, Domenico Amalfitano University of Naples Federico II, Anna Rita Fasolino Federico II University of Naples, Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy
File Attached
11:30
15m
Short-paper
AI-Driven Machine Learning Architecture for Scalable Irrigation Detection in Precision Agriculture: A Case Study with CropXIndustry Track Paper
Industry Program
Jakub Ozimek University of Groningen, Jakub Ozimek University of Groningen, Andrea Capiluppi University of Groningen
11:45
15m
Short-paper
Exploring Architectural Smells Detection Through LLMsResearch Track Paper
Research Papers
Claudio Tessa University of Milano-Bicocca, Matteo Bochicchio University of Milano-Bicocca, Francesca Arcelli Fontana University of Milano-Bicocca
12:00
15m
Short-paper
LLM-based Quality Assessment of Software Architecture Diagrams: A Preliminary Study with Four Open-Source ProjectsResearch Track Paper
Research Papers
Glauber Oliveira University of Fortaleza, Nabor Mendonca University of Fortaleza