LogExpert: Log-based Recommended Resolutions Generation using Large Language Model
Software logs play a vital role in ensuring the reliability and availability of large-scale software systems. In recent years, researchers have made significant efforts to build log analysis approaches to manage software systems. However, these approaches focus on log compression, log parsing and log anomaly detection. In the current context, engineers continue to spend substantial time and effort on resolving errors once anomalous logs have been detected. To achieve truly automated software system management and high-level Artificial Intelligence for IT Operations (AIOps), it’s necessary to bridge the gap between anomalous logs and their resolutions.
In this paper, we propose a novel framework LogExpert to automatically generate recommended resolutions for anomalous logs. Specifically, we build a log recognizer to utilize the wealth of software knowledge in technical forums such as Stack Overflow (SO). In addition, LogExpert combines the great power of a Large Language Model (LLM) with domain-specific knowledge to generate the resolution. We conducted a preliminary evaluation of our framework on datasets from SO. Our log recognizer achieves the F1 score of 0.936. Our lexical metrics and human evaluation show the overall LogExpert framework achieves excellent performance in log-based resolution generation. \footnote{Our code and datasets are available in the supplemental material.}
Thu 18 AprDisplayed time zone: Lisbon change
11:00 - 12:30 | LLM, NN and other AI technologies 3New Ideas and Emerging Results / Research Track / Software Engineering Education and Training / Software Engineering in Practice at Pequeno Auditório Chair(s): Tushar Sharma Dalhousie University | ||
11:00 15mTalk | Xpert: Empowering Incident Management with Query Recommendations via Large Language Models Research Track Yuxuan Jiang University of Michigan Ann-Arbor, Chaoyun Zhang Microsoft, Shilin He Microsoft Research, Zhihao Yang Peking University, Minghua Ma Microsoft Research, Si Qin Microsoft Research, Yu Kang Microsoft Research, Yingnong Dang Microsoft Azure, Saravan Rajmohan Microsoft 365, Qingwei Lin Microsoft, Dongmei Zhang Microsoft Research | ||
11:15 15mTalk | Tensor-Aware Energy Accounting Research Track DOI Pre-print | ||
11:30 15mTalk | LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systems Software Engineering in Practice Mohamad Fakih University of California, Irvine, Rahul Dharmaji University of California, Irvine, Yasamin Moghaddas University of California, Irvine, Gustavo Quiros Siemens Technology, Tosin Ogundare Siemens Technology, Mohammad Al Faruque UCI | ||
11:45 15mTalk | Resolving Code Review Comments with Machine Learning Software Engineering in Practice Alexander Frömmgen Google, Jacob Austin Google, Peter Choy Google, Nimesh Ghelani Google, Lera Kharatyan Google, Gabriela Surita Google, Elena Khrapko Google, Pascal Lamblin Google, Pierre-Antoine Manzagol Google, Marcus Revaj Google, Maxim Tabachnyk Google, Danny Tarlow Google, Kevin Villela Google, Dan Zheng Google DeepMind, Satish Chandra Google, Inc, Petros Maniatis Google DeepMind | ||
12:00 15mTalk | LLMs Still Can't Avoid Instanceof: An investigation Into GPT-3.5, GPT-4 and Bard's Capacity to Handle Object-Oriented Programming Assignments Software Engineering Education and Training | ||
12:15 7mTalk | Leveraging Large Language Models to Improve REST API Testing New Ideas and Emerging Results Myeongsoo Kim Georgia Institute of Technology, Tyler Stennett Georgia Institute of Technology, Dhruv Shah Georgia Institute of Technology, Saurabh Sinha IBM Research, Alessandro Orso Georgia Institute of Technology Pre-print | ||
12:22 7mTalk | LogExpert: Log-based Recommended Resolutions Generation using Large Language Model New Ideas and Emerging Results JiaboWang Beijing University of Posts and Telecommunications, guojun chu Beijing University of Posts and Telecommunications, Jingyu Wang , Haifeng Sun Beijing University of Posts and Telecommunications, Qi Qi , Yuanyi Wang Beijing University of Posts and Telecommunications, Ji Qi China Mobile (Suzhou) Software Technology Co., Ltd., Jianxin Liao Beijing University of Posts and Telecommunications |