Evaluating Cross-Language Transfer for Refactoring Detection with Large Language Models
The ability to detect refactoring is crucial to understanding software evolution. Many studies have proposed automated refactoring detection tools, such as RefactoringMiner and PyRef, but most are language-specific. Developing tools for new languages requires creating custom rules for each language, which is time-consuming and labor-intensive. This paper empirically evaluates the potential of Cross-Language Transfer (CLT) Learning using Large Language Models (LLMs) as a more general and cost-effective alternative for refactoring detection. Specifically, we compare CLT prompting against zero-shot prompting and native few-shot prompting. In CLT prompting, examples from one language (Java) are used to detect refactorings in another language (Python). In native few-shot prompting, examples are written in the same language as the target. Our results show that cross-language (Java→Python) transfer is highly effective, significantly outperforming the zero-shot prompting (Avg. F1 0.501 vs. 0.430). Surprisingly, it nearly matches the performance of native few-shot prompting (Python→Python, Avg. F1 0.506). These findings suggest that CLT is a promising direction for bypassing language-specific tool development.
Wed 18 MarDisplayed time zone: Athens change
14:00 - 15:30 | Session 2A - Refactoring, Code Smells, and Software MaintenanceResearch Track / Early Research Achievement (ERA) Track / Journal First Track / Industrial Track / Short Papers and Posters Track at Panorama Chair(s): Stefan Grintz SAP | ||
14:00 15mTalk | An Empirical Analysis of Code Clones in GitHub Actions Workflows Research Track Guillaume Cardoen University of Mons, Tom Mens University of Mons, Alexandre Decan University of Mons; F.R.S.-FNRS | ||
14:15 15mTalk | Reusing Legacy Code in Wasm: Key Challenges of Compilation and Code Semantics Preservation Research Track Sara Baradaran University of Southern California, Liyan Huang University of Southern California, Mukund Raghothaman University of Southern California, Weihang Wang University of Southern California Pre-print | ||
14:30 15mTalk | Cold-Start Anti-Patterns and Refactorings in Serverless Systems: An Empirical Study Research Track SYED SALAUDDIN MOHAMMAD TARIQ University of Michigan - Dearborn, Foyzul Hassan University of Michigan at Dearborn, Amiangshu Bosu Wayne State University, Probir Roy University of Michigan at Dearborn | ||
14:45 15mTalk | Prescriptive procedure for manual code smell annotation Journal First Track Simona Prokić Faculty of Technical Sciences, University of Novi Sad, Nikola Luburić Faculty of Technical Sciences, University of Novi Sad, Jelena Slivka Faculty of Technical Sciences, University of Novi Sad, Aleksandar Kovačević Faculty of Technical Sciences, University of Novi Sad | ||
15:00 15mTalk | Transpilation using Recursive Rewrite Rules: From Legacy to Maintainable Code Industrial Track Tristan Albers , Jos Hegge , Pierre van de Laar TNO-ESI, Niels Brouwers , Paul Nelissen , Wilbert Alberts , George Azis , Theo Baan , Danny Handoko , Quint van der Linden | ||
15:15 7mTalk | Evaluating Cross-Language Transfer for Refactoring Detection with Large Language Models Early Research Achievement (ERA) Track Siyuan Liu Nara Institute of Science and Technology, Nabhan Suwanachote Nara Institute of Science and Technology, Yutaro Kashiwa Nara Institute of Science and Technology, Brittany Reid Nara Institute of Science and Technology, Hajimu Iida Nara Institute of Science and Technology | ||
15:22 7mTalk | Design Pattern-based Code Refactoring with LLMs Short Papers and Posters Track Bartolomeo Zisa Università di Pisa, Lucia Passaro University of Pisa, Jacopo Soldani University of Pisa, Italy | ||