Reducing Events to Augment Log-based Anomaly Detection Models: An Empirical Study
As software systems grow increasingly intricate, the precise detection of anomalies have become both essential and challenging. Current log-based anomaly detection methods depend heavily on vast amounts of log data leading to inefficient inference and potential misguidance by noise logs. However, the quantitative effects of log reduction on the effectiveness of anomaly detection remain unexplored. Therefore, we first conduct a comprehensive study on six distinct models spanning three datasets. Through the study, the impact of log quantity and their effectiveness in representing anomalies is qualifies, uncovering three distinctive log event types that differently influence model performance. Drawing from these insights, we propose LogCleaner: an efficient methodology for the automatic reduction of log events in the context of anomaly detection. Serving as middleware between software systems and models, LogCleaner continuously updates and filters anti-events and duplicative-events in the raw generated logs. Experimental outcomes highlight LogCleaner’s capability to reduce over 70% of log events in anomaly detection, accelerating the model’s inference speed by approximately 300%, and universally improving the performance of models for anomaly detection.
Thu 24 OctDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
14:00 - 15:30 | Repository miningESEM Journal-First Papers / ESEM IGC / ESEM Technical Papers at Multimedia (B3 Building - Hall) Chair(s): Apostolos Ampatzoglou University of Macedonia | ||
14:00 20mFull-paper | Decoding Android Permissions: A Study of Developer Challenges and Solutions on Stack Overflow ESEM Technical Papers Sahrima Jannat Oishwee University of Saskatchewan, Zadia Codabux University of Saskatchewan, Natalia Stakhanova University of Saskatchewan | ||
14:20 20mFull-paper | Negative Results of Image Processing for Identifying Duplicate Questions on Stack Overflow ESEM Technical Papers | ||
14:40 20mFull-paper | Understanding Fairness in Software Engineering: Insights from Stack Exchange Sites ESEM Technical Papers Emeralda Sesari University of Groningen, Federica Sarro University College London, Ayushi Rastogi University of Groningen, The Netherlands DOI Pre-print | ||
15:00 15mIndustry talk | Reducing Events to Augment Log-based Anomaly Detection Models: An Empirical Study ESEM IGC Lingzhe Zhang Peking University, China, Tong Jia Institute for Artificial Intelligence, Peking University, Beijing, China, Kangjin Wang Alibaba Group, Mengxi Jia Peking University, Yong Yang , Ying Li School of Software and Microelectronics, Peking University, Beijing, China | ||
15:15 15mJournal Early-Feedback | The upper bound of information diffusion in code review ESEM Journal-First Papers Michael Dorner Blekinge Institute of Technology, Daniel Mendez Blekinge Institute of Technology and fortiss, Krzysztof Wnuk , Ehsan Zabardast Blekinge Institute of Technology, Jacek Czerwonka Developer Services, Microsoft Link to publication DOI Pre-print |