Enhancing GitHub Actions Failure Explanations: Log Preprocessing and Prompt Optimization with LLMs
Large Language Models (LLMs) have emerged as a promising tool for automating CI/CD debugging. However, LLM-based failure diagnosis in GitHub Actions (GA) workflows is often time-consuming. This study examines the effectiveness of different LLMs for diagnosing GA workflow failures. First, we introduce a novel approach for log processing that systematically filters and structures raw log data, significantly enhancing the quality of input for LLM. Second, we conduct a comprehensive evaluation of several enhanced prompt engineering strategies, including zero-shot, few-shot and self-refinement, to identify the most effective methods for generating accurate and actionable failure explanations. Our experiments with 100 GitHub GitHub Actions workflow logs demonstrate the promise of our proposed solution, showing substantial improvements in correctness, relevance, and clarity of failure diagnoses.
Fri 20 FebDisplayed time zone: Chennai, Kolkata, Mumbai, New Delhi change
11:15 - 12:40 | |||
11:15 20mResearch paper | Proactive Software Fault Prediction with LLMs, Balanced ML, and Postmortem Analysis Research Papers Monika Yadav Central University of Haryana, Mahendergarh, Lov Kumar National Institute of Technology, Kurukshetra, Vishal Passricha Central University of Haryana, Mahendergarh | ||
11:35 20mResearch paper | Enhancing GitHub Actions Failure Explanations: Log Preprocessing and Prompt Optimization with LLMs Research Papers Venkata Sai Sravya Sambaturu University of Michigan-Flint, Meriem Mastouri University of Michigan-Flint, Belhassen Khefacha University of Michigan-Flint, Ameen Vathimattom Ashraf University of Michigan-Flint, Rafique Agyare University of Michigan-Flint, Mohamed Wiem Mkaouer University of Michigan-Flint | ||
11:55 20mResearch paper | Refining Tests through API Response Evaluation Research Papers | ||
12:15 12mShort-paper | kS-LLM: k-Step based Automatic LLM Test Case Generator using Caching Mechanism to Achieve Higher Code Coverage Research Papers Anand Sharma National Institute of Technology Warangal, VIVEK YELLETI National Institute of Technology Warangal, Sangharatna Godboley National Institute of Technology Warangal, P. Radha Krishna National Institute of Technology Warangal, Warangal | ||
12:27 12mShort-paper | A Data-Driven Framework for Evaluating Mobile UI Usability through Interaction Elements and Design Semantics Research Papers Gundala Shanmukhi Rama National Institute of Technology Warangal, Sangharatna Godboley National Institute of Technology Warangal, RAVICHANDRA SADAM National Institute of Technology Warangal | ||