Logging, which aims to determine the position of logging statements, the verbosity levels, and the log messages, is a crucial process for software reliability enhancement. In recent years, numerous automatic logging tools have been designed to assist developers in one of the logging tasks (\eg providing suggestions on whether to log in try-catch blocks). These tools are useful in certain situations yet cannot provide a comprehensive logging solution in general. Moreover, although recent research has started to explore end-to-end logging, it is still largely constrained by the high cost of fine-tuning, hindering its practical usefulness in software development. To address these problems, this paper proposes UniLog, an automatic logging framework based on the in-context learning (ICL) paradigm of large language models (LLMs). Specifically, UniLog can perform in-context inference with only a prompt containing five demonstration examples without any model tuning. In addition, UniLog can further enhance its logging ability after warmup with only a few hundred random samples. We evaluated UniLog on a large dataset containing 12,012 code snippets extracted from 1,465 GitHub repositories. The results show that UniLog achieved the state-of-the-art performance in automatic logging: (1) 76.9% accuracy in selecting logging positions, (2) 72.3% accuracy in predicting verbosity levels, and (3) 27.1 BLEU-4 score in generating log messages. Meanwhile, UniLog requires less than 4% of the parameter tuning time needed by fine-tuning the same LLM.
Wed 17 AprDisplayed time zone: Lisbon change
14:00 - 15:30 | LLM, NN and other AI technologies 1Journal-first Papers / Research Track / New Ideas and Emerging Results at Luis de Freitas Branco Chair(s): Shin Yoo Korea Advanced Institute of Science and Technology | ||
14:00 15mTalk | EGFE: End-to-end Grouping of Fragmented Elements in UI Designs with Multimodal Learning Research Track Liuqing Chen Zhejiang University, Yunnong Chen Zhejiang University, Shuhong Xiao , Yaxuan Song Zhejiang University, Lingyun Sun Zhejiang University, Yankun Zhen Alibaba Group, Tingting Zhou Alibaba Group, Yanfang Chang Alibaba Group Link to publication Pre-print Media Attached File Attached | ||
14:15 15mTalk | A Comprehensive Study of Learning-based Android Malware Detectors under Challenging Environments Research Track Gao Cuiying Huazhong University of Science and Technology, Gaozhun Huang Huazhong University of Science and Technology, Heng Li Huazhong University of Science and Technology, Bang Wu Huazhong University of Science and Technology, Yueming Wu Nanyang Technological University, Wei Yuan Huazhong University of Science and Technology | ||
14:30 15mTalk | Toward Automatically Completing GitHub Workflows Research Track Antonio Mastropaolo Università della Svizzera italiana, Fiorella Zampetti University of Sannio, Italy, Gabriele Bavota Software Institute @ Università della Svizzera Italiana, Massimiliano Di Penta University of Sannio, Italy Pre-print | ||
14:45 15mTalk | UniLog: Automatic Logging via LLM and In-Context Learning Research Track Junjielong Xu The Chinese University of Hong Kong, Shenzhen, Ziang Cui Southeast University, Yuan Zhao Peking University, Xu Zhang Microsoft Research, Shilin He Microsoft Research, Pinjia He Chinese University of Hong Kong, Shenzhen, Liqun Li Microsoft Research, Yu Kang Microsoft Research, Qingwei Lin Microsoft, Yingnong Dang Microsoft Azure, Saravan Rajmohan Microsoft 365, Dongmei Zhang Microsoft Research | ||
15:00 7mTalk | Self-Supervised Learning to Prove Equivalence Between Straight-Line Programs via Rewrite Rules Journal-first Papers Steve Kommrusch Leela AI, Martin Monperrus KTH Royal Institute of Technology, Louis-Noël Pouchet Colorado State University | ||
15:07 7mTalk | NLP-based Automated Compliance Checking of Data Processing Agreements against GDPR Journal-first Papers Orlando Amaral University of Luxembourg, Muhammad Ilyas Azeem University of Luxembourg, Sallam Abualhaija University of Luxembourg, Lionel Briand University of Ottawa, Canada; Lero centre, University of Limerick, Ireland | ||
15:14 7mTalk | Exploring ChatGPT for Toxicity Detection in GitHub New Ideas and Emerging Results |