LogReducer: Identify and Reduce Log Hotspots in Kernel on the Fly
Modern systems generate a massive amount of logs to detect and diagnose system faults, which incurs expensive storage cost and runtime overhead. After investigating real-world production logs, we observe that most of the logging overhead is due to a small number of log templates, referred to as log hotspots. Therefore, we conduct a systematical study about log hotspots in an industrial system $\mathcal{W}$, which motivates us to identify log hotspots and reduce them on the fly. In this paper, we propose \textit{LogReducer}, a non-intrusive and language-independent log reduction framework based on eBPF (Extended Berkeley Packet Filter), consisting of both online and offline processes. After two months of serving the offline process of \textit{LogReducer} in $\mathcal{W}$, the log storage overhead has dropped from 19.7 PB per day to 12.0 PB (i.e., about a 39.08% decrease). Practical implementation and experimental evaluations in the test environment demonstrate that the online process of \textit{LogReducer} can control the logging overhead of hotspots while preserving logging effectiveness. Moreover, the log hotspot handling time can be reduced from average 9 days in production to 10 minutes in the test with the help of \textit{LogReducer}.
Fri 19 MayDisplayed time zone: Hobart change
11:00 - 12:30 | Runtime analysis and self-adaptationTechnical Track / NIER - New Ideas and Emerging Results / SEIP - Software Engineering in Practice / Journal-First Papers at Level G - Plenary Room 1 Chair(s): Domenico Bianculli University of Luxembourg | ||
11:00 15mTalk | Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attention Technical Track Cheryl Lee The Chinese University of Hong Kong, Tianyi Yang The Chinese University of Hong Kong, Zhuangbin Chen Chinese University of Hong Kong, China, Yuxin Su Sun Yat-sen University, Yongqiang Yang Huawei Technologies, Michael Lyu The Chinese University of Hong Kong Pre-print | ||
11:15 15mTalk | Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models Technical Track Toufique Ahmed University of California at Davis, Supriyo Ghosh Microsoft, Chetan Bansal Microsoft Research, Thomas Zimmermann Microsoft Research, Xuchao Zhang Microsoft, Saravanakumar Rajmohan Microsoft 365 Pre-print | ||
11:30 15mTalk | Eadro: An End-to-End Troubleshooting Framework for Microservices on Multi-source Data Technical Track Cheryl Lee The Chinese University of Hong Kong, Tianyi Yang The Chinese University of Hong Kong, Zhuangbin Chen Chinese University of Hong Kong, China, Yuxin Su Sun Yat-sen University, Michael Lyu The Chinese University of Hong Kong Pre-print | ||
11:45 15mTalk | LogReducer: Identify and Reduce Log Hotspots in Kernel on the Fly Technical Track Guangba Yu Sun Yat-Sen University, Pengfei Chen Sun Yat-Sen University, Pairui Li Tencent Inc., Tianjun Weng Tencent Inc., Haibing Zheng Tencent, Yuetang Deng Tencent, Zibin Zheng School of Software Engineering, Sun Yat-sen University Pre-print | ||
12:00 15mTalk | TraceArk: Towards Actionable Performance Anomaly Alerting for Online Service Systems SEIP - Software Engineering in Practice Zhengran Zeng Southern University of Science and Technology, Yuqun Zhang Southern University of Science and Technology, Yong Xu Microsoft Research, Minghua Ma Microsoft Research, Bo Qiao Microsoft Research, Wentao Zou , Qingjun Chen , Meng Zhang , Xu Zhang Microsoft Research, Hongyu Zhang The University of Newcastle, Xuedong Gao , Hao Fan , Saravan Rajmohan Microsoft 365, Qingwei Lin Microsoft Research, Dongmei Zhang Microsoft Research | ||
12:15 7mTalk | ActivFORMS: A Formally-Founded Model-Based Approach to Engineer Self-Adaptive Systems Journal-First Papers | ||
12:22 7mTalk | Auto-Logging: AI-centred Logging Instrumentation NIER - New Ideas and Emerging Results Pre-print |