ASE 2024
Sun 27 October - Fri 1 November 2024 Sacramento, California, United States
Wed 30 Oct 2024 11:00 - 11:15 at Carr - Log and trace; failure and fault Chair(s): Yiming Tang

Producing accurate software models is crucial in model-driven software engineering (MDE). However, modeling complex systems is an error-prone task that requires deep application domain knowledge. In the past decade, several automated techniques have been pro- posed to support academic and industrial practitioners by providing relevant modeling operations. Nevertheless, those techniques require a huge amount of training data that cannot be available due to several factors, e.g., privacy issues. The advent of large language models (LLMs) can support the generation of synthetic data although state-of-the-art approaches are not yet supporting the generation of modeling operations. To fill the gap, we propose a conceptual framework that combines modeling event logs, intelligent modeling assistants, and the generation of modeling operations using LLMs. In particular, the architecture comprises modeling components that help the designer specify the system, record its operation within a graphical modeling environment, and automatically recommend relevant operations. In addition, we generate a completely new dataset of modeling events by telling on the most prominent LLMs currently available. As a proof of concept, we instantiate the proposed framework using a set of existing modeling tools employed in industrial use cases within different European projects. To assess the proposed methodology, we first evaluate the capability of the examined LLMs to generate realistic modeling operations by relying on well-founded distance metrics. Then, we evaluate the recommended operations by considering real-world industrial modeling artifacts. Our findings demonstrate that LLMs can generate modeling events even though the overall accuracy is higher when considering human-based operations. In this respect, we see generative AI tools as an alternative when the modeling operations are not available to train traditional IMAs specifically conceived to support industrial practitioners.

Wed 30 Oct

Displayed time zone: Pacific Time (US & Canada) change

10:30 - 12:00
Log and trace; failure and faultResearch Papers / Industry Showcase at Carr
Chair(s): Yiming Tang Rochester Institute of Technology
10:30
15m
Talk
Demonstration-Free: Towards More Practical Log Parsing with Large Language Models
Research Papers
Yi Xiao , Van-Hoang Le The University of Newcastle, Hongyu Zhang Chongqing University
10:45
15m
Talk
Unlocking the Power of Numbers: Log Compression via Numeric Token Parsing
Research Papers
Siyu Yu The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), Yifan Wu Peking University, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Pinjia He Chinese University of Hong Kong, Shenzhen
11:00
15m
Talk
Towards Synthetic Trace Generation of Modeling Operations using In-Context Learning Approach
Research Papers
Vittoriano Muttillo University of Teramo, Claudio Di Sipio University of l'Aquila, Riccardo Rubei University of L'Aquila, Luca Berardinelli Johannes Kepler University Linz, MohammadHadi Dehghani Johannes Kepler University Linz
11:15
15m
Talk
DeployFix: Dynamic Repair of Software Deployment Failures via Constraint Solving
Industry Showcase
Haoyu Liao East China Normal University, Jianmei Guo East China Normal University, Bo Huang East China Normal University, Yujie Han East China Normal University, Dingyu Yang Zhejiang University, Kai Shi Alibaba Group, Jonathan Ding Intel, Guoyao Xu Alibaba Group, Guodong Yang Alibaba Group, Liping Zhang Alibaba Group
11:30
15m
Talk
FAIL: Analyzing Software Failures from the News Using LLMs
Research Papers
Dharun Anandayuvaraj Purdue University, Matthew Campbell Purdue University, Arav Tewari Purdue University, James C. Davis Purdue University
DOI Pre-print
11:45
15m
Talk
Do not neglect what's on your hands: localizing software faults with exception trigger streamACM SigSoft Distinguished Paper Award
Research Papers
Xihao Zhang School of Computer Science, Wuhan University, Yi Song School of Computer Science, Wuhan University, Xiaoyuan Xie Wuhan University, Qi Xin Wuhan University, Chenliang Xing School of Computer Science, Wuhan University