Exploring Large Language Models for Analyzing and Improving Method Names in Scientific Code
Research scientists increasingly rely on implementing software to support their research. While previous research has examined the impact of identifier names on program comprehension in traditional programming environments, limited work has explored this area in scientific software, especially regarding the quality of method names in the code. The recent advances in Large Language Models (LLMs) present new opportunities for automating code analysis tasks, such as identifier name appraisals and recommendations. Our study evaluates four popular LLMs on their ability to analyze grammatical patterns and suggest improvements for 496 method names extracted from Python-based Jupyter Notebooks. Our findings show that the LLMs are somewhat effective in analyzing these method names and generally follow good naming practices, like starting method names with verbs. However, their inconsistent handling of domain-specific terminology and only moderate agreement with human annotations indicate that automated suggestions require human evaluation. This work provides foundational insights for improving the quality of scientific code through AI automation.
Fri 3 OctDisplayed time zone: Hawaii change
15:40 - 17:00 | Program Comprehension and Review 2ESEM - Emerging Results and Vision Track / at Kaiulani II Chair(s): Chris Brown Virginia Tech | ||
15:40 26mTalk | Dealing with SonarQube Cloud: Initial Results from a Mining Software Repository Study ESEM - Emerging Results and Vision Track Sabato Nocera University of Salerno, Davide Fucci Blekinge Institute of Technology, Giuseppe Scanniello University of Salerno | ||
16:06 26mTalk | Exploring Large Language Models for Analyzing and Improving Method Names in Scientific Code ESEM - Emerging Results and Vision Track Gunnar Larsen University of Hawaii at Manoa, Carol Wong University of Hawaii at Manoa, Anthony Peruma University of Hawai‘i at Mānoa Pre-print | ||
16:33 26mTalk | Identifier Name Similarities: An Exploratory Study ESEM - Emerging Results and Vision Track Carol Wong University of Hawaii at Manoa, Mai Abe University of Hawai‘i at Mānoa, Silvia De Benedictis University of Hawai‘i at Mānoa, Marissa Halim University of Hawai‘i at Mānoa, Anthony Peruma University of Hawai‘i at Mānoa | ||