InsightAI: Root Cause Analysis in Large Hierarchical Log Files with Private Data Using Large Language Models
This program is tentative and subject to change.
Abstract—[Problem] As industries increasingly depend on complex software systems, efficient log analysis is essential for maintaining reliability and privacy. However, Identifying problems through logs is often time-consuming and costly for developers. [Background] Large language models (LLMs) can automate parts of log analysis, but challenges like limited computational resources and the frequent need to retrain LLMs due to the dynamic nature of software logs persist. External LLMs, such as GPTs, along with in-context learning techniques, can help reduce some of these issues, but other challenges, including token limitations, high token costs, and data privacy, remain. [Method] To tackle these challenges, we developed an automated pipeline that extracts log files and employs in-context learning, allowing the model to efficiently adapt to changes without extensive retraining. Our approach introduces a novel flame-graph-like method that reduces token usage, thereby lowering token-related costs and response latency while maintaining high accuracy. [Results] This solution allows industries to automate log analysis, minimize system downtime, and enhance performance, all while keeping data privacy and maintaining operational efficiency. [Conclusion] Our flame-graph-like methodology reduces input tokens by 93.61% and processing latency by 77.45%. Our anonymization results show an improvement of 138.63% over the baseline. This industrial experience report presents our approach to allow industries to balance token costs, maintain response accuracy, and ensure data privacy while relying on external LLMs without the need to manage computational resources directly.
This program is tentative and subject to change.
Sun 27 AprDisplayed time zone: Eastern Time (US & Canada) change
14:00 - 15:30 | |||
14:00 15mTalk | Themes of Building LLM-based Applications for Production: A Practitioner's View Research and Experience Papers Alina Mailach Leipzig University, Sebastian Simon Leipzig University, Johannes Dorn Leipzig University, Norbert Siegmund Leipzig University | ||
14:15 15mTalk | LLM-Based Safety Case Generation for Baidu Apollo: Are We There Yet? Research and Experience Papers | ||
14:30 15mTalk | An AI-driven Requirements Engineering Framework Tailored for Evaluating AI-Based Software Research and Experience Papers Hamed Barzamini , Fatemeh Nazaritiji Northern Illinois University, Annalise Brockmann Northern Illinois University, Hasan Ferdowsi Northern Illinois university, Mona Rahimi Northern Illinois University | ||
14:46 14mTalk | Engineering LLM Powered Multi-agent Framework for Autonomous CloudOps Research and Experience Papers Kannan Parthasarathy MontyCloud, Karthik Vaidhyanathan IIIT Hyderabad, Rudra Dhar SERC, IIIT Hyderabad, India, Venkat Krishnamachari MontyCloud, Adyansh Kakran International Institute of Information Technology, Hyderabad, Sreemaee Akshathala IIIT Hyderabad, Shrikara Arun IIIT Hyderabad, Amey Karan IIIT Hyderabad, Basil Muhammed MontyCloud, Sumant Dubey MontyCloud, Mohan Veerubhotla MontyCloud | ||
15:00 15mTalk | Generating and Verifying Synthetic Datasets with Requirements Engineering Research and Experience Papers Lynn Vonderhaar Embry-Riddle Aeronautical University, Timothy Elvira Embry-Riddle Aeronautical University, Omar Ochoa Embry-Riddle Aeronautical University | ||
15:15 15mTalk | InsightAI: Root Cause Analysis in Large Hierarchical Log Files with Private Data Using Large Language Models Research and Experience Papers Maryam Ekhlasi Polytechnique Montreal, Anurag Prakash Ciena, Michel Dagenais Polytechnique Montréal, Maxime Lamothe Polytechnique Montreal |