Cross-System Software Log-based Anomaly Detection Using Meta-Learning
Modern software systems produce vast amounts of logs, serving as an essential resource for anomaly detection. Artificial Intelligence for IT Operations (AIOps) tools have been developed to automate the process of log-based anomaly detection for software systems. However, no single tool is designed to address these practical challenges together: high data labeling costs, evolving logs in dynamic systems, and adaptability across different systems. In this paper, we propose CroSysLog, an AIOps tool for log-event level anomaly detection, specifically designed to address these practical challenges. Following prior studies, CroSysLog uses a neural representation approach to gain a nuanced understanding of logs and generate representations for individual log events accordingly. CroSysLog can be trained on source systems with sufficient labeled log events from open datasets to achieve robustness, and then efficiently adapt to target systems with a few labeled log events for effective anomaly detection. We evaluate CroSysLog using open datasets of four large-scale distributed supercomputing systems: BGL, Thunderbird, Liberty, and Spirit. We used random log splits, maintaining the chronological order of consecutive log events, from these systems to train and evaluate CroSysLog. Our results show that, after training CroSysLog on Liberty and BGL as source systems, CroSysLog can efficiently adapt to target systems Thunderbird and Spirit using a few labeled log events from each target system, effectively performing anomaly detection for these target systems. The results demonstrate that CroSysLog is a practical, scalable, and adaptable tool for log-event level anomaly detection in operational and maintenance contexts of software systems.
Thu 6 MarDisplayed time zone: Eastern Time (US & Canada) change
14:00 - 15:30 | Defect Prediction & AnalysisResearch Papers / Industrial Track / Journal First Track at L-1710 Chair(s): Rrezarta Krasniqi University of North Carolina at Charlotte | ||
14:00 15mTalk | An ensemble learning method based on neighborhood granularity discrimination index and its application in software defect prediction Research Papers Yuqi Sha College of Information Science and Technology,Qingdao University of Science and Technology, Feng Jiang College of Information Science and Technology,Qingdao University of Science and Technology, Qiang Hu College of Information Science and Technology, Qingdao University of Science and technology, Yifan He Institute of Cosmetic Regulatory Science,Beijing Technology and Business University | ||
14:15 15mTalk | ALOGO: A Novel and Effective Framework for Online Cross-project Defect Prediction Research Papers Rongrong Shi Beijing Jiaotong University, Yuxin He Beijing Jiaotong University, Ying Liu Beijing Jiaotong University, Zonghao Li Beijing Jiaotong University, Jingxin Su Beijing Jiaotong University, Haonan Tong Beijing Jiaotong University | ||
14:30 15mTalk | Cross-System Software Log-based Anomaly Detection Using Meta-Learning Research Papers Yuqing Wang University of Helsinki, Finland, Mika Mäntylä University of Helsinki and University of Oulu, Jesse Nyyssölä University of Helsinki, Ke Ping University of Helsinki, Liqiang Wang University of Wyoming Pre-print | ||
14:45 15mTalk | RADICE: Causal Graph Based Root Cause Analysis for System Performance Diagnostic Industrial Track Andrea Tonon Huawei Ireland Research Center, Meng Zhang Shandong University, Bora Caglayan Huawei Ireland Research Center, Fei Shen Huawei Nanjing Research Center, Tong Gui , Mingxue Wang Huawei Ireland Research Center, Rong Zhou | ||
15:00 15mTalk | Can We Trust the Actionable Guidance from Explainable AI Techniques in Defect Prediction? Research Papers | ||
15:15 15mTalk | Making existing software quantum safe: A case study on IBM Db2 Journal First Track Lei Zhang , Andriy Miranskyy Toronto Metropolitan University (formerly Ryerson University), Walid Rjaibi IBM Canada Lab, Greg Stager IBM Canada Lab, Michael Gray IBM, John Peck IBM |