Towards Synthetic Trace Generation of Modeling Operations using In-Context Learning Approach
Producing accurate software models is crucial in model-driven software engineering (MDE). However, modeling complex systems is an error-prone task that requires deep application domain knowledge. In the past decade, several automated techniques have been pro- posed to support academic and industrial practitioners by providing relevant modeling operations. Nevertheless, those techniques require a huge amount of training data that cannot be available due to several factors, e.g., privacy issues. The advent of large language models (LLMs) can support the generation of synthetic data although state-of-the-art approaches are not yet supporting the generation of modeling operations. To fill the gap, we propose a conceptual framework that combines modeling event logs, intelligent modeling assistants, and the generation of modeling operations using LLMs. In particular, the architecture comprises modeling components that help the designer specify the system, record its operation within a graphical modeling environment, and automatically recommend relevant operations. In addition, we generate a completely new dataset of modeling events by telling on the most prominent LLMs currently available. As a proof of concept, we instantiate the proposed framework using a set of existing modeling tools employed in industrial use cases within different European projects. To assess the proposed methodology, we first evaluate the capability of the examined LLMs to generate realistic modeling operations by relying on well-founded distance metrics. Then, we evaluate the recommended operations by considering real-world industrial modeling artifacts. Our findings demonstrate that LLMs can generate modeling events even though the overall accuracy is higher when considering human-based operations. In this respect, we see generative AI tools as an alternative when the modeling operations are not available to train traditional IMAs specifically conceived to support industrial practitioners.
Wed 30 OctDisplayed time zone: Pacific Time (US & Canada) change
10:30 - 12:00 | Log and trace; failure and faultResearch Papers / Industry Showcase at Carr Chair(s): Yiming Tang Rochester Institute of Technology | ||
10:30 15mTalk | Demonstration-Free: Towards More Practical Log Parsing with Large Language Models Research Papers | ||
10:45 15mTalk | Unlocking the Power of Numbers: Log Compression via Numeric Token Parsing Research Papers | ||
11:00 15mTalk | Towards Synthetic Trace Generation of Modeling Operations using In-Context Learning Approach Research Papers Vittoriano Muttillo University of Teramo, Claudio Di Sipio University of l'Aquila, Riccardo Rubei University of L'Aquila, Luca Berardinelli Johannes Kepler University Linz, MohammadHadi Dehghani Johannes Kepler University Linz | ||
11:15 15mTalk | DeployFix: Dynamic Repair of Software Deployment Failures via Constraint Solving Industry Showcase Haoyu Liao East China Normal University, Jianmei Guo East China Normal University, Bo Huang East China Normal University, Yujie Han East China Normal University, Dingyu Yang Zhejiang University, Kai Shi Alibaba Group, Jonathan Ding Intel, Guoyao Xu Alibaba Group, Guodong Yang Alibaba Group, Liping Zhang Alibaba Group | ||
11:30 15mTalk | FAIL: Analyzing Software Failures from the News Using LLMs Research Papers Dharun Anandayuvaraj Purdue University, Matthew Campbell Purdue University, Arav Tewari Purdue University, James C. Davis Purdue University DOI Pre-print | ||
11:45 15mTalk | Do not neglect what's on your hands: localizing software faults with exception trigger stream Research Papers Xihao Zhang School of Computer Science, Wuhan University, Yi Song School of Computer Science, Wuhan University, Xiaoyuan Xie Wuhan University, Qi Xin Wuhan University, Chenliang Xing School of Computer Science, Wuhan University |