CAShift: Benchmarking Log-Based Cloud Attack Detection under Normality Shift
With the rapid advancement of cloud-native computing, securing cloud environments has become an important task. Log-based Anomaly Detection (LAD) is the most representative technique used in different systems for attack detection and safety guarantee, where multiple LAD methods and relevant datasets have been proposed. However, even though some of these datasets are specifically prepared for cloud systems, they only cover limited cloud behaviors and lack information from a whole-system perspective. Another critical issue to consider is normality shift, which implies that the test distribution could differ from the training distribution and highly affect the performance of LAD. Unfortunately, existing works only focus on simple shift types such as chronological changes, while other important and cloud-specific shift types are ignored, e.g., the distribution shift introduced by different deployed cloud architectures. Therefore, creating a new dataset that covers diverse behaviors of cloud systems and normality shift types is necessary.
To fill this gap, we construct the first normality shift-aware dataset CAShift to evaluate the performance of LAD in cloud, which considers different roles of software in cloud systems, supports three real-world normality shift types (application shift, version shift, and cloud architecture shift), and features 20 different attack scenarios in various cloud system components. Based on CAShift, we conduct a comprehensive empirical study to investigate the effectiveness of existing LAD methods in normality shift scenarios. Additionally, to explore the feasibility of shift adaptation, we further investigate three continuous learning approaches, which are the most common methods to mitigate the impact of distribution shift. Results demonstrated that 1) all LAD methods suffer from normality shift where the performance drops up to 34%, and 2) existing continuous learning methods are promising to address shift drawbacks, but the ratio of data used for model retraining and the selection of algorithms highly affect the shift adaptation, with an increase in the F1-Score of up to 27%. Based on our findings, we offer valuable implications for future research in designing more robust LAD models and methods for LAD shift adaptation.
Tue 24 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
16:00 - 17:40 | Anomaly DetectionIdeas, Visions and Reflections / Research Papers / Industry Papers at Pirsenteret 150 Chair(s): Gias Uddin York University, Canada | ||
16:00 20mTalk | Cross-System Categorization of Abnormal Traces in Microservice-Based Systems via Meta-Learning Research Papers Yuqing Wang University of Helsinki, Finland, Mika Mäntylä University of Helsinki and University of Oulu, Serge Demeyer University of Antwerp and Flanders Make vzw, Mutlu Beyazıt University of Antwerp and Flanders Make vzw, Joanna Kisaakye University of Antwerp, Belgium, Jesse Nyyssölä University of Helsinki DOI | ||
16:20 10mTalk | CLSLog: Collaborating Large and Small Models for Log-based Anomaly Detection Ideas, Visions and Reflections Pei Xiao Peking University, Tong Jia Institute for Artificial Intelligence, Peking University, Beijing, China, Chiming Duan Peking University, Minghua He Peking University, Weijie Hong Peking university, Xixuan Yang School of Software and Microelectronics, Peking University, Yihan Wu National Computer Network Emergency Response Technical Team/Coordination Center of China, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Gang Huang Peking University | ||
16:30 10mTalk | From Few-Label to Zero-Label: An Approach for Cross-System Log-Based Anomaly Detection with Meta-Learning Ideas, Visions and Reflections Xinlong Zhao Peking University, Tong Jia Institute for Artificial Intelligence, Peking University, Beijing, China, Minghua He Peking University, Yihan Wu National Computer Network Emergency Response Technical Team/Coordination Center of China, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Gang Huang Peking University | ||
16:40 20mTalk | CAShift: Benchmarking Log-Based Cloud Attack Detection under Normality Shift Research Papers Jiongchi Yu Singapore Management University, Xiaofei Xie Singapore Management University, Qiang Hu Tianjin University, Bowen Zhang Singapore Management University, Ziming Zhao Zhejiang University, Yun Lin Shanghai Jiao Tong University, Lei Ma The University of Tokyo & University of Alberta, Ruitao Feng Southern Cross University, Frank Liauw Government Technology Agency Singapore DOI Pre-print | ||
17:00 20mTalk | Detecting and Handling WoT Violations by Learning Physical Interactions from Device Logs Research Papers Bingkun Sun Fudan University, Shiqi Sun Northwestern Polytechnique University, Jialin Ren Fudan University, Mingming Hu Fudan University, Kun Hu School of Computer Science, Fudan University, Liwei Shen Fudan University, Xin Peng Fudan University DOI | ||
17:20 20mTalk | L4: Diagnosing Large-scale LLM Training Failures via Automated Log Analysis Industry Papers Zhihan Jiang The Chinese University of Hong Kong, Junjie Huang The Chinese University of Hong Kong, Guangba Yu The Chinese University of Hong Kong, Zhuangbin Chen Sun Yat-sen University, Yichen LI The Chinese University of Hong Kong, Renyi Zhong The Chinese University of Hong Kong, Cong Feng Huawei Cloud Computing Technology, Yongqiang Yang Huawei Cloud Computing Technology, Zengyin Yang Computing and Networking Innovation Lab, Huawei Cloud Computing Technology Co., Ltd, Michael Lyu Chinese University of Hong Kong | ||
This room is located outside Clarion Hotel
This room is located in the Pirsenteret (The Pier Center) convention center. It is just outside the hotel, on the back, towards the fjord.
You should be able to go through the emergency exit at Clarion, just on the side of the Cosmos 3 wing, which will be bring you close to Pirsenteret.
The entrance to the center is from here:
https://maps.app.goo.gl/dU3qH6kAimXGBNHe7
Once inside, go all straight and you will find signage to reach the room. The room is known as room 150 inside the center.