SmartNote: An LLM-Powered, Personalised Release Note Generator That Just Works
The release note is a crucial document outlining changes in new software versions. It plays a key role in helping stakeholders recognise important changes and understand the implications behind them. Despite this fact, many developers view the process of writing software release notes as a tedious and dreadful task. Consequently, numerous tools (e.g., DeepRelease and Conventional Changelog) have been developed by researchers and practitioners to automate the generation of software release notes. However, these tools fail to consider project domain and target audience for personalisation, limiting their relevance and conciseness. Additionally, they suffer from limited applicability, often necessitating significant workflow adjustments and adoption efforts, hindering practical use and stressing developers. Despite recent advancements in natural language processing and the proven capabilities of large language models (LLMs) in various code and text-related tasks, there are no existing studies investigating the integration and utilisation of LLMs in automated release note generation. Therefore, we propose SmartNote, a novel and widely applicable release note generation approach that produces high-quality, contextually personalised release notes using LLM technology. SmartNote aggregates changes and uses an LLM to describe and summarise the changes using code, commit, and pull request details. It categorises and scores (for significance) commits to generate structured and concise release notes of prioritised changes. Our human and automatic evaluations reveal that SmartNote outperforms or achieves comparable performance to DeepRelease (state-of-the-art), Conventional Changelog (off-the-shelf), and the projects’ original release notes across four quality metrics: completeness, clarity, conciseness, and organisation. In both evaluations, SmartNote ranked first for completeness and organisation, while clarity ranked first in the human evaluation. A further evaluation demonstrates that SmartNote is effective in terms of context awareness and applicability.
Tue 24 JunDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
10:30 - 12:20 | Code Review, Build, and ReleaseIdeas, Visions and Reflections / Industry Papers / Demonstrations / Research Papers / Journal First at Aurora A Chair(s): Peter Rigby Concordia University; Meta | ||
10:30 10mTalk | From Overload to Insight: Bridging Code Search and Code Review with LLMs Ideas, Visions and Reflections Nikitha Rao Carnegie Mellon University, Bogdan Vasilescu Carnegie Mellon University, Reid Holmes University of British Columbia | ||
10:40 20mTalk | Explaining Explanations: An Empirical Study of Explanations in Code Reviews Journal First Ratnadira Widyasari Singapore Management University, Singapore, Ting Zhang Singapore Management University, Abir Bouraffa University of Hamburg, Walid Maalej University of Hamburg, David Lo Singapore Management University | ||
11:00 10mTalk | Support, Not Automation: Towards AI-supported Code Review for Code Quality and Beyond Ideas, Visions and Reflections | ||
11:10 20mTalk | BitsAI-CR: Automated Code Review via LLM in Practice Industry Papers Tao Sun Beihang University, Jian Xu ByteDance, Yuanpeng Li ByteDance, Zhao Yan ByteDance, Ge Zhang ByteDance, Lintao Xie ByteDance, Lu Geng ByteDance, Zheng Wang University of Leeds, Yueyan Chen ByteDance, Qin Lin ByteDance, Wenbo Duan ByteDance, Kaixin Sui ByteDance, Yuanshuo Zhu ByteDance | ||
11:30 10mTalk | Visualising Developer Interactions in Code Reviews Demonstrations | ||
11:40 20mTalk | CXXCrafter: An LLM-Based Agent for Automated C/C++ Open Source Software Building Research Papers Zhengmin Yu Fudan University, Yuan Zhang Fudan University, Ming Wen Huazhong University of Science and Technology, Yinan Nie Fudan University, Zhang Wenhui Fudan University, Min Yang Fudan University DOI | ||
12:00 20mTalk | SmartNote: An LLM-Powered, Personalised Release Note Generator That Just Works Research Papers Farbod Daneshyan Peking University, Runzhi He Peking University, Jianyu Wu Peking University, Minghui Zhou Peking University DOI |
Aurora A is the first room in the Aurora wing.
When facing the main Cosmos Hall, access to the Aurora wing is on the right, close to the side entrance of the hotel.