Human-written API documentation often becomes outdated, requiring developers to update it manually. Researchers have proposed identifying outdated API references in documentation, yet have not addressed updating API documentation. Now, emerging large language models (LLMs) are capable of generating code examples and text descriptions. Then, a key question arises: Can LLMs assist in updating API documentation? In this paper, we propose an approach for leveraging an LLM to update API documentation with code change information. To evaluate this approach, we select five open-source projects that manage documentation revisions on GitHub and analyze the differences in documentation between two releases to derive ground truths. We then assess the accuracy of LLM-generated updates by comparing them to the ground truths. Our results show that LLM-generated updates achieve higher METEOR than outdated API documentation (0.776 vs 0.679). It indicates that the LLM updates are more similar to the human updates than the outdated documentation. Our results also reveal that LLM correctly updates code-related information in API documentation.
Wed 10 SepDisplayed time zone: Auckland, Wellington change
10:30 - 12:00 | Session 1 - DocumentationResearch Papers Track / Industry Track / Registered Reports at Case Room 3 260-055 Chair(s): Ashkan Sami Edinburgh Napier University | ||
10:30 15m | APIDocBooster: An Extract-Then-Abstract Framework Leveraging Large Language Models for Augmenting API Documentation Research Papers Track Chengran Yang Singapore Management University, Singapore, Jiakun Liu Harbin Institute of Technology, Bowen Xu North Carolina State University, Christoph Treude Singapore Management University, Yunbo Lyu Singapore Management University, Junda He Singapore Management University, Ming Li Nanjing University, David Lo Singapore Management University | ||
10:45 15m | Automatically Augmenting GitHub Issues with Informative User Reviews Research Papers Track Arthur Pilone University of São Paulo, Marco Raglianti Software Institute - USI, Lugano, Michele Lanza Software Institute - USI, Lugano, Fabio Kon University of São Paulo, Paulo Meirelles University of São Paulo Pre-print | ||
11:00 15m | Can LLMs Update API Documentation? Research Papers Track Seonah Lee Gyeongsang National University, Jueun Heo , Katherine R. Dearstyne University of Notre Dame | ||
11:15 15m | RMGenie: An LLM-Based Agent Framework for Open Source Software README Generation Research Papers Track Xing Cui Institute of Software, Chinese Academy of Sciences, Jingzheng Wu Institute of Software, The Chinese Academy of Sciences, Zhiyuan Li , Tianyue Luo (Institute of Software Chinese Academy of Sciences), Xiang Ling Institute of Software, Chinese Academy of Sciences | ||
11:30 15m | Requirements Ambiguity Detection and Explanation with LLMs: An Industrial Study Industry Track Sarmad Bashir RISE Research Institutes of Sweden, Alessio Ferrari Consiglio Nazionale delle Ricerche (CNR) and University College Dublin (UCD), Muhammad Abbas Khan RISE Research Institutes of Sweden, Per Erik Strandberg Westermo Network Technologies AB, Zulqarnain Haider Alstom Rail AB, Sweden, Mehrdad Saadatmand RISE Research Institutes of Sweden, Markus Bohlin Mälardalen University Pre-print | ||
11:45 10m | Learning From the Best: What Makes Popular Hugging Face Models? A Registered Report Registered Reports | ||