ICSE 2024
Fri 12 - Sun 21 April 2024 Lisbon, Portugal
Thu 18 Apr 2024 14:15 - 14:30 at Pequeno Auditório - LLM, NN and other AI technologies 4 Chair(s): Oscar Chaparro

Recently, deep learning models have been widely applied in program understanding tasks, and these models achieve state-of-the-art results on many benchmark datasets. A major challenge of deep learning for program understanding is that the effectiveness of these approaches depends on the quality of their datasets, and these datasets often contain noisy data samples. A typical kind of noise in program understanding datasets is label noise, which means that the target outputs for some inputs are incorrect.

Researchers have proposed various approaches to alleviate the negative impact of noisy labels, and formed a new research topic: noisy label learning (NLL). In this paper, we conduct an empirical study on the effectiveness of noisy label learning on deep learning for program understanding datasets. We evaluate various NLL approaches and deep learning models on three tasks: program classification, vulnerability detection, and code summarization. From the evaluation results, we come to the following findings: 1) small trained-from-scratch models are prone to label noises in program understanding, while large pre-trained models are highly robust against them. 2) NLL approaches significantly improve the program classification accuracies for small models on noisy training sets, but they only slightly benefit large pre-trained models in classification accuracies. 3) NLL can effectively detect synthetic noises in program understanding, but struggle in detecting real-world noises. We believe our findings can provide insights on the abilities of NLL in program understanding, and shed light on future works in tackling noises in software engineering datasets.

Thu 18 Apr

Displayed time zone: Lisbon change

14:00 - 15:30
LLM, NN and other AI technologies 4Research Track / Industry Challenge Track / New Ideas and Emerging Results at Pequeno Auditório
Chair(s): Oscar Chaparro William & Mary
14:00
15m
Talk
Programming Assistant for Exception Handling with CodeBERT
Research Track
Yuchen Cai University of Texas at Dallas, Aashish Yadavally The University of Texas at Dallas, Abhishek Mishra University of Texas at Dallas, Genesis Montejo University of Texas at Dallas, Tien N. Nguyen University of Texas at Dallas
14:15
15m
Talk
An Empirical Study on Noisy Label Learning for Program Understanding
Research Track
Wenhan Wang Nanyang Technological University, Yanzhou Li Nanyang Technological University, Anran Li Nanyang Technological University, Jian Zhang Nanyang Technological University, Wei Ma Nanyang Technological University, Singapore, Yang Liu Nanyang Technological University
Pre-print
14:30
15m
Talk
An Empirical Study on Low GPU Utilization of Deep Learning Jobs
Research Track
Yanjie Gao Microsoft Research, yichen he , Xinze Li Microsoft Research, Bo Zhao Microsoft Research, Haoxiang Lin Microsoft Research, Yoyo Liang Microsoft, Jing Zhong Microsoft, Hongyu Zhang Chongqing University, Jingzhou Wang Microsoft Research, Yonghua Zeng Microsoft, Keli Gui Microsoft, Jie Tong Microsoft, Mao Yang Microsoft Research
DOI
14:45
15m
Talk
Using an LLM to Help With Code Understanding
Research Track
Daye Nam Carnegie Mellon University, Andrew Macvean Google, Inc., Vincent J. Hellendoorn Carnegie Mellon University, Bogdan Vasilescu Carnegie Mellon University, Brad A. Myers Carnegie Mellon University
15:00
15m
Talk
MissConf: LLM-Enhanced Reproduction of Configuration-Triggered Bugs
Industry Challenge Track
Ying Fu National University of Defense Technology, Teng Wang National University of Defense Technology, Shanshan Li National University of Defense Technology, Jinyan Ding National University of Defense Technolog, Shulin Zhou National University of Defense Technology, Zhouyang Jia National University of Defense Technology, Wang Li National University of Defense Technology, Yu Jiang Tsinghua University, Xiangke Liao National University of Defense Technology
15:15
7m
Talk
XAIport: A Service Framework for the Early Adoption of XAI in AI Model Development
New Ideas and Emerging Results
Zerui Wang Concordia University, Yan Liu Concordia University, Abishek Arumugam Thiruselvi Concordia University, Wahab Hamou-Lhadj Concordia University, Montreal, Canada
DOI Pre-print
15:22
7m
Talk
Which Syntactic Capabilities Are Statistically Learned by Masked Language Models for Code?
New Ideas and Emerging Results
Alejandro Velasco William & Mary, David Nader Palacio William & Mary, Daniel Rodriguez-Cardenas , Denys Poshyvanyk William & Mary
Pre-print
Hide past events