Automatically Generating Descriptive Texts in Logging Statements: How Far Are We?
In most cases, logs are the only accurate information available for administrators to understand system behavior and diagnose failure root causes. However, due to the lack of well-defined logging guidance, it is challenging for developers to decide what to log, especially logging statements that contain descriptive texts and variables. In this paper, we explore automatically generation of descriptive texts in logging statements and evaluate the effectiveness of various automatic generation methods. We propose that to generate descriptive texts in logging statements can be transferred as a retrieval-based Q&A task. According to the roles of query and answer, we design two retrieval strategies including Code&Code and Code&Log. To measure the similarity between the query and answer, we utilize two types of retrieval algorithms including Information retrieval-based and neural networks-based algorithms. We conduct a systematic analysis of various retrieval algorithms under different retrieval strategies in terms of their effectiveness, and assess their accuracy using the automatic metrics and human evaluation during which 5 instructive findings are presented. We believe that these findings can provide potential implications for both researchers and practitioners for relevant research. Moreover, we construct and release a log text dataset containing over 138K valid log texts from 85 Java projects in Apache ecosystem for future logging statement analysis and generation.
Tue 1 DecDisplayed time zone: Osaka, Sapporo, Tokyo change
16:00 - 17:30 | Program Generation, Transactions and AutomationResearch Papers at online Chair(s): Shigeru Chiba The University of Tokyo | ||
16:00 30mTalk | Automatically Generating Descriptive Texts in Logging Statements: How Far Are We? Research Papers Xiaotong Liu School of Software and Microelectronics, Peking University, Beijing, China, Tong Jia School of Software and Microelectronics, Peking University, Beijing, China, Ying Li School of Software and Microelectronics, Peking University, Beijing, China, Hao Yu Peking University, Yang Yue University of California, Irvine, Chuanjia Hou School of Software and Microelectronics, Peking University, Beijing, China | ||
16:30 30mTalk | Banyan: Coordination-free Transactions over Mergeable Replicated Data Types Research Papers Shashank Shekhar Dubey IIT Madras, KC Sivaramakrishnan IIT Madras, Thomas Gazagnaire Tarides, Anil Madhavapeddy University of Cambridge | ||
17:00 30mTalk | Stack-Driven Program Generation of WebAssembly Research Papers |