Personalized Code Readability Assessment: Are We There Yet?
This program is tentative and subject to change.
Unreadable code could be a breeding ground for errors. Thus, previous work defined approaches based on machine learning to automatically assess code readability that can warn developers when some code artifacts (e.g., classes) become unreadable. Given datasets of code snippets manually evaluated by several developers in terms of their perceived readability, such approaches (i) establish a snippet-level ground truth, and (ii) train a binary (readable/unreadable) or a ternary (readable/neutral/unreadable) code readability classifier. Given this procedure, all existing approaches neglect the subjectiveness of code readability, i.e., the possible different developer-specific nuances in the code readability perception. In this paper, we aim to understand to what extent it is possible to assess code readability as subjectively perceived by developers through a personalized code readability assessment approach. This problem is significantly more challenging than the snippet-level classification problem: We assume that, in a realistic scenario, a given developer is keen to provide only a few code readability evaluations, thus less data is available. For this reason, we adopt an LLM with few-shot learning to achieve our goal. Our results, however, show that such an approach achieves worse results than a state-of-the-art feature-based model that is trained to work at the snippet-level. We tried to understand why this happens by looking more closely at the quality of the available code readability datasets and assessed the consistency of the inter-developer evaluations. We observed that up to a third of the evaluations are self-contradictory. Our negative results call for new and more reliable code readability datasets.
This program is tentative and subject to change.
Sun 27 AprDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | Vulnerabilities, Technical Debt, DefectsEarly Research Achievements (ERA) / Research Track / Replications and Negative Results (RENE) at 205 | ||
11:00 10mTalk | CalmDroid: Core-Set Based Active Learning for Multi-Label Android Malware Detection Research Track Minhong Dong Tiangong University, Liyuan Liu Tiangong University, Mengting Zhang Tiangong University, Sen Chen Tianjin University, Wenying He Hebei University of Technology, Ze Wang Tiangong University, Yude Bai Tianjin University | ||
11:10 10mTalk | Towards Task-Harmonious Vulnerability Assessment based on LLM Research Track Zaixing Zhang Southeast University, Chang Jianming , Tianyuan Hu Southeast University, Lulu Wang Southeast University, Bixin Li Southeast University | ||
11:20 10mTalk | Slicing-Based Approach for Detecting and Patching Vulnerable Code Clones Research Track Hakam W. Alomari Miami University, Christopher Vendome Miami University, Himal Gyawali Miami University | ||
11:30 6mTalk | Revisiting Security Practices for GitHub Actions Workflows Early Research Achievements (ERA) | ||
11:36 6mTalk | Leveraging multi-task learning to improve the detection of SATD and vulnerability Replications and Negative Results (RENE) Barbara Russo Free University of Bolzano, Jorge Melegati Free University of Bozen-Bolzano, Moritz Mock Free University of Bozen-Bolzano Pre-print | ||
11:42 10mTalk | Leveraging Context Information for Self-Admitted Technical Debt Detection Research Track Miki Yonekura Nara Institute of Science and Technology, Yutaro Kashiwa Nara Institute of Science and Technology, Bin Lin Hangzhou Dianzi University, Kenji Fujiwara Nara Women’s University, Hajimu Iida Nara Institute of Science and Technology | ||
11:52 6mTalk | Personalized Code Readability Assessment: Are We There Yet? Replications and Negative Results (RENE) Antonio Vitale Politecnico di Torino, University of Molise, Emanuela Guglielmi University of Molise, Rocco Oliveto University of Molise, Simone Scalabrino University of Molise | ||
11:58 6mTalk | Automated Refactoring of Non-Idiomatic Python Code: A Differentiated Replication with LLMs Replications and Negative Results (RENE) Pre-print | ||
12:04 10mResearch paper | Sonar: Detecting Logic Bugs in DBMS through Generating Semantic-aware Non-Optimizing Query Research Track Shiyang Ye Zhejiang University, Chao Ni Zhejiang University, Jue Wang Nanjing University, Qianqian Pang zhejang university, Xinrui Li School of Software Technology, Zhejiang University, xiaodanxu College of Computer Science and Technology, Zhejiang university | ||
12:14 6mTalk | A Study on Applying Large Language Models to Issue Classification Replications and Negative Results (RENE) | ||
12:20 10mLive Q&A | Session's Discussion: "Vulnerabilities, Technical Debt, Defects" Research Track |