ICPC 2024
Sun 14 - Sat 20 April 2024 Lisbon, Portugal
co-located with ICSE 2024

Automated log analysis is crucial in modern software-intensive systems for facilitating program comprehension throughout software maintenance and engineering life cycles. Existing methods perform tasks such as log parsing and log anomaly detection by providing a single prediction value without interpretation. However, given the increasing volume of system events, the limited interpretability of analysis results hinders analysts’ comprehension of program status and their ability to take appropriate actions. Moreover, these methods require substantial in-domain training data, and their performance declines sharply (by up to 62.5%) in online scenarios involving unseen logs from new domains, a common occurrence due to rapid software updates. In this paper, we propose LogPrompt, a novel interpretable log analysis approach for online scenarios. LogPrompt employs large language models (LLMs) to perform online log analysis tasks via a suite of advanced prompt strategies tailored for log tasks, which enhances LLMs’ performance by up to 380.7% compared with simple prompts. Experiments on nine publicly available evaluation datasets across two tasks demonstrate that LogPrompt, despite requiring no in-domain training, outperforms existing approaches trained on thousands of logs by up to 55.9%. We also conduct a human evaluation of LogPrompt’s interpretability, with six practitioners possessing over 10 years of experience, who highly rated the generated content in terms of usefulness and readability (averagely 4.42/5). LogPrompt also exhibits remarkable compatibility with open-source and smaller-scale LLMs, making it flexible for practical deployment. We release the code of LogPrompt.

Mon 15 Apr

Displayed time zone: Lisbon change

11:00 - 12:30
11:00
10m
Talk
Towards Summarizing Code Snippets Using Pre-Trained TransformersICPCICPC Full paper
Research Track
Antonio Mastropaolo Università della Svizzera italiana, Matteo Ciniselli Università della Svizzera Italiana, Luca Pascarella ETH Zurich, Rosalia Tufano Università della Svizzera Italiana, Emad Aghajani Software Institute, USI Università della Svizzera italiana, Gabriele Bavota Software Institute @ Università della Svizzera Italiana
Pre-print
11:10
10m
Talk
Generating Java Methods: An Empirical Assessment of Four AI-Based Code AssistantsICPCICPC Full paper
Research Track
Vincenzo Corso University of Milano - Bicocca, Leonardo Mariani University of Milano-Bicocca, Daniela Micucci University of Milano-Bicocca, Italy, Oliviero Riganelli University of Milano - Bicocca
Pre-print
11:20
10m
Talk
Analyzing Prompt Influence on Automated Method Generation: An Empirical Study with CopilotICPCICPC Full paper
Research Track
Ionut Daniel Fagadau University of Milano - Bicocca, Leonardo Mariani University of Milano-Bicocca, Daniela Micucci University of Milano-Bicocca, Italy, Oliviero Riganelli University of Milano - Bicocca
Pre-print
11:30
10m
Talk
Interpretable Online Log Analysis Using Large Language Models with Prompt StrategiesICPCICPC Full paper
Research Track
Yilun Liu Huawei co. LTD, Shimin Tao University of Science and Technology of China; Huawei co. LTD, Weibin Meng Huawei co. LTD, Jingyu Wang , Wenbing Ma Huawei co. LTD, Yuhang Chen University of Science and Technology of China, Yanqing Zhao Huawei co. LTD, Hao Yang Huawei co. LTD, Yanfei Jiang Huawei co. LTD
Pre-print
11:40
10m
Talk
Do Machines and Humans Focus on Similar Code? Exploring Explainability of Large Language Models in Code SummarizationICPCICPC RENE Paper
Replications and Negative Results (RENE)
Jiliang Li Vanderbilt University, Yifan Zhang Vanderbilt University, Zachary Karas Vanderbilt University, Collin McMillan University of Notre Dame, Kevin Leach Vanderbilt University, Yu Huang Vanderbilt University
Pre-print
11:50
10m
Talk
Knowledge-Aware Code Generation with Large Language ModelsICPCICPC Full paper
Research Track
Tao Huang Shandong Normal University, Zhihong Sun Shandong Normal University, Zhi Jin Peking University, Ge Li Peking University, Chen Lyu Shandong Normal University
Pre-print
12:00
8m
Talk
Enhancing Source Code Representations for Deep Learning with Static AnalysisICPCICPC ERA Paper
Early Research Achievements (ERA)
Xueting Guan University of Melbourne, Christoph Treude Singapore Management University
Pre-print
12:08
8m
Talk
AthenaLLM: Supporting Experiments with Large Language Models in Software DevelopmentICPCICPC Tools
Tool Demonstration
Benedito Fernando Albuquerque de Oliveira Federal University of Pernambuco, Fernando Castor University of Twente and Federal University of Pernambuco
12:16
14m
Talk
AI-Assisted Program Comprehension: Panel with SpeakersICPC
Discussion