Towards Secure Logging: Characterizing and Benchmarking Logging Code Security Issues with LLMs
Logging code plays an important role in software systems by recording key events and behaviors, which are essential for debugging and monitoring. However, insecure logging practices can inadvertently expose sensitive information or enable attacks such as log injection, posing serious threats to system security and privacy. Prior research has examined general defects in logging code, but systematic analysis of logging code security issues remains limited, particularly in leveraging LLMs for detection and repair. In this paper, we derive a comprehensive taxonomy of logging code security issues, encompassing four common issue categories and 10 corresponding patterns. We further construct a benchmark dataset with 101 real-world logging security issue reports that have been manually reviewed and annotated. We then propose an automated framework that incorporates various contextual knowledge to evaluate LLMs’ capabilities in detecting and repairing logging security issues. Our experimental results reveal a notable disparity in performance: while LLMs are moderately effective at detecting security issues (e.g., the accuracy ranges from 12.9% to 52.5% on average), they face noticeable challenges in reliably generating correct code repairs. We also find that the issue description alone improves the LLMs’ detection accuracy more than the security pattern explanation or a combination of both. Overall, our findings provide actionable insights for practitioners and highlight the potential and limitations of current LLMs for secure logging.
Tue 7 JulDisplayed time zone: Eastern Time (US & Canada) change
14:00 - 15:20 | LoggingJournal-First Paper / Research Papers at MB 3.445 Chair(s): Xiaoyin Wang University of Texas at San Antonio | ||
14:00 20mTalk | Small is Beautiful: A Practical and Efficient Log Parsing Framework Research Papers Minxing Wang Singapore Management University, Yintong Huo Singapore Management University, Singapore Pre-print | ||
14:20 20mTalk | Towards Secure Logging: Characterizing and Benchmarking Logging Code Security Issues with LLMs Research Papers He Yang Yuan York University, Xin Wang The Hong Kong University of Science and Technology (Guangzhou), Kundi Yao Ontario Tech University, An Ran Chen University of Alberta, Zishuo Ding The Hong Kong University of Science and Technology (Guangzhou), Zhenhao Li York University Pre-print | ||
14:40 20mTalk | Enhancing Log Sentiments: An Exploratory Study of Sentiments and Emotions with Software Logs Journal-First Paper Xiaohui Wang University of Waterloo, Youshuai Tan Macau University of Science and Technology, Zishuo Ding The Hong Kong University of Science and Technology (Guangzhou), Jinfu Chen Wuhan University, Jifeng Xuan Wuhan University, Weiyi Shang University of Waterloo | ||
15:00 20mTalk | LPB-Gen: Systematic Large Log-Parsing Benchmarks Generation Journal-First Paper Hetong Dai University of Waterloo, Kundi Yao Ontario Tech University, Felix Li University of Waterloo, Jianxin You University of Montreal, Qianyun Shen University of Montreal, Weiyi Shang University of Waterloo | ||