Model Transformations Using LLMs Out-of-the-Box: Can Accidental Complexity Be Reduced?
Model-driven engineering envisions an enhancement of software engineering by promoting automation through model transformations. However, the effective use of model-driven tools often requires significant expertise due to their reliance on custom domain-specific languages for transformations. This expertise gap, combined with challenges like inadequate tool support and the need for additional training, has meant that model-driven engineering sometimes struggled to reduce, and might have even increased, accidental complexity. Addressing this problem, our work investigates the use of large language models, specifically ChatGPT-4, to reduce accidental complexity in model transformation processes within model-driven engineering. We conducted a systematic literature review and designed an experiment to explore ChatGPT-4’s efficacy in performing model transformations out-of-the-box. Using a semi-automated pipeline, we applied ChatGPT-4 to 99 UML class diagram models, generating Java programs and comparing them with ground truth programs created by a state-of-the-art modelling tool. Our findings indicate a cumulative success rate of 94% after three iterations, with most generation errors being resolved during the process. However, complex models presented a significant challenge, with a cumulative success rate of only 17%
Thu 12 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
09:00 - 10:30 | |||
09:00 30mKeynote | LLMs for Software Engineering: What does that mean for model-driven software development? LLM4SE Gabriele Taentzer Philipps-Universität Marburg | ||
09:30 30mResearch paper | Provider-Agnostic Knowledge Graph Extraction from User Stories using Large Language Models LLM4SE Thayna Camargo da SIlva , Leen Lambers Brandenburg University of Technology Cottbus-Senftenberg, Sébastien Mosser McMaster University, Kate Revoredo Humboldt-Universität zu Berlin | ||
10:00 30mResearch paper | Model Transformations Using LLMs Out-of-the-Box: Can Accidental Complexity Be Reduced? LLM4SE Gabriel Kazai Mälardalen University, Ronnie Agyeiwaa Osei Mälardalen University, Antonio Cicchetti Mälardalen University, Alessio Bucaioni Mälardalen University |