Diagram-Aware Automatic Review of Software Design Documents Using Multimodal Large Language Models
With the growing capabilities of Large Language Models (LLMs), their application to software engineering tasks has garnered increasing attention. While our previous work focused on automated review of software design documents composed of text and tables, diagram-based artifacts such as UML or screen transition diagrams remained outside the scope of analysis. Reviewing such documents requires accurate interpretation of structural and semantic elements, including nodes, edges, and transition conditions. This paper proposes a hybrid diagram-understanding method that leverages multimodal LLMs together with diagram structure extracted from Office Open XML (OOXML) representations, guided by Chain-of-Thought prompting, to incrementally interpret diagrams in real-world design documents. The method enables integrated analysis across diagrams, tables, and textual content within the review process. Evaluation results show that the proposed approach achieves high accuracy in tasks involving structural recognition, such as consistency checking. However, the results also highlight persistent challenges in semantic-level review tasks, particularly in interpreting complex transition conditions. These findings provide insights into the current capabilities and limitations of multimodal LLMs for high-stakes design document reviews.
Fri 20 MarDisplayed time zone: Athens change
11:00 - 12:30 | Session 6A - Tools and Techniques for Effective Software DevelopmentIndustrial Track / Journal First Track / Tool Demo Track / Research Track at Panorama Chair(s): NIKIEMA Beninwende Serge Lionel University of Luxembourg | ||
11:00 15mTalk | How Natural Language Proficiency Shapes GenAI Code for Software Engineering Tasks Journal First Track Ruksit Rojpaisarnkit Nara Institute of Science and Technology, Youmei Fan Nara Institute of Science and Technology, Kenichi Matsumoto Nara Institute of Science and Technology, Raula Gaikovina Kula The University of Osaka | ||
11:15 15mTalk | Data Catalog Tools: A Systematic Multivocal Literature Review Journal First Track Marco Tonnarelli JADS - TU/e, Indika Kumara Tilburg University, Stefan Driessen JADS, Tilburg University, Damian Andrew Tamburri University of Sannio - JADS/NXP Semiconductors, Willem-Jan van den Heuvel JADS, Tilburg University, Patrick Oor NXP Semiconductors | ||
11:30 15mTalk | On the Practical Adoption of a Static Performance Anti-Pattern Detector: An Industrial Case Study Industrial Track Lizhi Liao University of Guelph, Weiyi Shang University of Waterloo, Catalin Sporea ERA Environmental Management Solutions, Andrei Toma ERA Environmental Management Solutions, Sarah Sajedi ERA Environmental Management Solutions | ||
11:45 15mTalk | Multi-CoLoR: Context-Aware Localization and Reasoning across Multi-Language Codebases Industrial Track Indira Vats University of Toronto; Advanced Micro Devices (AMD), Sanjukta De Advanced Micro Devices, Subhayan Roy , Saurabh Bodhe , Lejin Varghese , Max Kiehn , Yonas Bedasso Advanced Micro Devices, Marsha Chechik University of Toronto Pre-print | ||
12:00 15mTalk | Diagram-Aware Automatic Review of Software Design Documents Using Multimodal Large Language Models Industrial Track | ||
12:15 7mTalk | Source Code-Driven GDPR Documentation: Supporting RoPA with Assessor View Tool Demo Track Mugdha Khedkar Heinz Nixdorf Institute, Paderborn University, Michael Schlichtig Heinz Nixdorf Institut, Paderborn University, Eric Bodden Heinz Nixdorf Institute at Paderborn University & Fraunhofer IEM Pre-print Media Attached | ||
12:22 7mTalk | RefineID: A Developer-Centric IDE Assistant for Better Identifiers Tool Demo Track | ||