Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction
Log parsing transforms log messages into structured formats, serving as a crucial step for log analysis. Despite a variety of log parsers that have been proposed, their performance on evolving log data remains unsatisfactory due to reliance on human-crafted rules or learning-based models with limited training data. The recent emergence of large language models (LLMs) has demonstrated strong abilities in understanding natural language and code, making it promising to apply LLMs for log parsing. Consequently, several studies have proposed LLM-based log parsers. However, LLMs may produce inaccurate templates, and existing LLM-based log parsers directly use the template generated by the LLM as the parsing result, hindering the accuracy of log parsing. Furthermore, these log parsers depend heavily on historical log data as demonstrations, which poses challenges in maintaining accuracy when dealing with scarce historical log data or evolving log data. To address these challenges, we propose AdaParser, an effective and adaptive log parsing framework using LLMs with self-generated in-context learning (SG-ICL) and self-correction. To facilitate accurate log parsing, AdaParser incorporates a novel component, a template corrector, which utilizes the LLM to correct potential parsing errors in the templates it generates. In addition, AdaParser maintains a dynamic candidate set composed of previously generated templates as demonstrations to adapt evolving log data. Extensive experiments on public large-scale datasets indicate that AdaParser outperforms state-of-the-art methods across all metrics, even in zero-shot scenarios. Moreover, when integrated with different LLMs, AdaParser consistently enhances the performance of the utilized LLMs by a large margin.
Mon 28 AprDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 17:30 | Log Parsing, Bug Localisation, Review ComprehensionResearch Track / Early Research Achievements (ERA) at 205 Chair(s): Gabriele Bavota Software Institute @ Università della Svizzera Italiana, Coen De Roover Vrije Universiteit Brussel, Gema Rodríguez-Pérez Department of Computer Science, Mathematics, Physics and Statistics, University of British Columbia, Okanagan Campus | ||
16:00 10mTalk | Developing a Taxonomy for Advanced Log Parsing Techniques Research Track Issam Sedki Concordia University, Wahab Hamou-Lhadj Concordia University, Montreal, Canada, Otmane Ait-Mohamed Concordia University, Naser Ezzati Jivan | ||
16:10 10mTalk | GELog:A GPT-Enhanced Log Representation Method for Anomaly Detection Research Track Wenwu Xu Institute of Information Engineering, Chinese Academy of Sciences and School of Cyberspace Security, University of Chinese Academy of Sciences, Peng Wang Institute of Information Engineering,Chinese Academy of Sciences, Haichao Shi Institute of Information Engineering,Chinese Academy of Sciences, Guoqiao Zhou Institute of Information Engineering,Chinese Academy of Sciences, Junliang Yao Institute of Information Engineering,Chinese Academy of Sciences, Xiao-Yu Zhang Institute of Information Engineering, Chinese Academy of Science | ||
16:20 10mTalk | Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction Research Track Yifan Wu Peking University, Siyu Yu The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), Ying Li School of Software and Microelectronics, Peking University, Beijing, China Pre-print | ||
16:30 10mTalk | LLM-BL: Large Language Models are Zero-Shot Rankers for Bug Localization Research Track Zhengliang Li Nanjing University, Zhiwei Jiang Nanjing University, Qiguo Huang NanJing Audit University, Qing Gu Nanjing University | ||
16:40 10mTalk | Improved IR-based Bug Localization with Intelligent Relevance Feedback Research Track Pre-print | ||
16:50 6mTalk | Towards Enhancing IR-based Bug Localization Leveraging Texts and Multimedia from Bug Reports Early Research Achievements (ERA) Shamima Yeasmin University of Saskatchewan, Chanchal K. Roy University of Saskatchewan, Kevin Schneider University of Saskatchewan, Masud Rahman Dalhousie University, Kartik Mittal University of Saskatchewan, Ryder Hardy University of Saskatchewan Pre-print | ||
16:56 10mTalk | Building Bridges, Not Walls: Fairness-aware and Accurate Recommendation of Code Reviewers via LLM-based Agents Collaboration Research Track Luqiao Wang Xidian University, Qingshan Li Xidian University, Di Cui Xidian University, Mingkang Wang Xidian University, Yutong Zhao University of Central Missouri, Yongye Xu Xidian University, Huiying Zhuang Xidian University, Yangtao Zhou Xidian University, Lu Wang Xidian University | ||
17:06 10mTalk | Code Review Comprehension: Reviewing Strategies Seen Through Code Comprehension Theories Research Track Pavlina Wurzel Goncalves University of Zurich, Pooja Rani University of Zurich, Margaret-Anne Storey University of Victoria, Diomidis Spinellis Athens University of Economics and Business & Delft University of Technology, Alberto Bacchelli University of Zurich Pre-print | ||
17:16 10mTalk | KotSuite: Unit Test Generation for Kotlin Programs in Android Applications Research Track Feng Yang Wuhan University, Qi Xin Wuhan University, Zhilei Ren Dalian University of Technology, Jifeng Xuan Wuhan University | ||
17:26 4mLive Q&A | Session's Discussion: "Log Parsing, Bug Localisation, Review Comprehension" Research Track |