LogExpert: Log-based Recommended Resolutions Generation using Large Language Model
Software logs play a vital role in ensuring the reliability and availability of large-scale software systems. In recent years, researchers have made significant efforts to build log analysis approaches to manage software systems. However, these approaches focus on log compression, log parsing and log anomaly detection. In the current context, engineers continue to spend substantial time and effort on resolving errors once anomalous logs have been detected. To achieve truly automated software system management and high-level Artificial Intelligence for IT Operations (AIOps), it’s necessary to bridge the gap between anomalous logs and their resolutions.
In this paper, we propose a novel framework LogExpert to automatically generate recommended resolutions for anomalous logs. Specifically, we build a log recognizer to utilize the wealth of software knowledge in technical forums such as Stack Overflow (SO). In addition, LogExpert combines the great power of a Large Language Model (LLM) with domain-specific knowledge to generate the resolution. We conducted a preliminary evaluation of our framework on datasets from SO. Our log recognizer achieves the F1 score of 0.936. Our lexical metrics and human evaluation show the overall LogExpert framework achieves excellent performance in log-based resolution generation. \footnote{Our code and datasets are available in the supplemental material.}
Thu 18 AprDisplayed time zone: Lisbon change
| 11:00 - 12:30 | LLM, NN and other AI technologies 3New Ideas and Emerging Results / Research Track / Software Engineering Education and Training / Software Engineering in Practice at Pequeno Auditório Chair(s): Tushar Sharma Dalhousie University | ||
| 11:0015m Talk | Xpert: Empowering Incident Management with Query Recommendations via Large Language Models Research Track Yuxuan Jiang University of Michigan Ann-Arbor, Chaoyun Zhang Microsoft, Shilin He Microsoft Research, Zhihao Yang Peking University, Minghua Ma Microsoft Research, Si Qin Microsoft Research, Yu Kang Microsoft Research, Yingnong Dang Microsoft Azure, Saravan Rajmohan Microsoft 365, Qingwei Lin Microsoft, Dongmei Zhang Microsoft Research | ||
| 11:1515m Talk | Tensor-Aware Energy Accounting Research TrackDOI Pre-print | ||
| 11:3015m Talk | LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systems Software Engineering in Practice Mohamad Fakih University of California, Irvine, Rahul Dharmaji University of California, Irvine, Yasamin Moghaddas University of California, Irvine, Gustavo Quiros Siemens Technology, Tosin Ogundare Siemens Technology, Mohammad Al Faruque UCI | ||
| 11:4515m Talk | Resolving Code Review Comments with Machine Learning Software Engineering in Practice Alexander Frömmgen Google, Jacob Austin Google, Peter Choy Google, Nimesh Ghelani Google, Lera Kharatyan Google, Gabriela Surita Google, Elena Khrapko Google, Pascal Lamblin Google, Pierre-Antoine Manzagol Google, Marcus Revaj Google, Maxim Tabachnyk Google, Danny Tarlow Google, Kevin Villela Google, Dan Zheng Google DeepMind, Satish Chandra Google, Inc, Petros Maniatis Google DeepMind | ||
| 12:0015m Talk | LLMs Still Can't Avoid Instanceof: An investigation Into GPT-3.5, GPT-4 and Bard's Capacity to Handle Object-Oriented Programming Assignments Software Engineering Education and Training | ||
| 12:157m Talk | Leveraging Large Language Models to Improve REST API Testing New Ideas and Emerging Results Myeongsoo Kim Georgia Institute of Technology, Tyler Stennett Georgia Institute of Technology, Dhruv Shah Georgia Institute of Technology, Saurabh Sinha IBM Research, Alessandro Orso Georgia Institute of TechnologyPre-print | ||
| 12:227m Talk | LogExpert: Log-based Recommended Resolutions Generation using Large Language Model New Ideas and Emerging Results JiaboWang  Beijing University of Posts and Telecommunications, guojun chu Beijing University of Posts and Telecommunications, Jingyu Wang , Haifeng Sun Beijing University of Posts and Telecommunications, Qi Qi , Yuanyi Wang Beijing University of Posts and Telecommunications, Ji Qi China Mobile (Suzhou) Software Technology Co., Ltd., Jianxin Liao Beijing University of Posts and Telecommunications | ||