Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia
Fri 19 May 2023 16:30 - 16:45 at Meeting Room 103 - Pre-trained and few shot learning for SE Chair(s): Yiling Lou

Large language models trained on massive code corpora can generalize to new tasks without the need for task-specific fine-tuning. In few-shot learning, these models take as input a prompt, composed of natural language instructions, a few instances of task demonstration, and a query and generate an output. However, the creation of an effective prompt for code-related tasks in few-shot learning has received little attention. We present a retrieval-based technique for composing the ingredients of a prompt effectively. We apply our approach, CEDAR, to two different programming languages, statically and dynamically typed, and two different tasks, namely, assertion generation and program repair. For each task, we compare CEDAR with state-of-the-art task-specific and fine-tuned models. Our empirical results show that, with only a few code example demonstrations, our retrieval-based code demonstration selection is effective in both tasks, with an accuracy of 76% and 52% for exact matches in test assertion generation and program repair tasks, respectively. For assertion generation, CEDAR outperforms existing task-specific and fine-tuned models by 333% and 11%, respectively, and for program repair, CEDAR yields 189% better accuracy than task-specific models and is competitive with recent fine-tuned models. These findings have practical implications for practitioners, as CEDAR could potentially be applied to multilingual and multitask settings without task or language-specific training with minimal examples and effort.

Fri 19 May

Displayed time zone: Hobart change

15:45 - 17:15
Pre-trained and few shot learning for SETechnical Track / Journal-First Papers at Meeting Room 103
Chair(s): Yiling Lou Fudan University
15:45
15m
Talk
On the validity of pre-trained transformers for natural language processing in the software engineering domain
Journal-First Papers
Alexander Trautsch University of Passau, Julian von der Mosel , Steffen Herbold University of Passau
16:00
15m
Talk
Automating Code-Related Tasks Through Transformers: The Impact of Pre-training
Technical Track
Rosalia Tufano Università della Svizzera Italiana, Luca Pascarella ETH Zurich, Gabriele Bavota Software Institute, USI Università della Svizzera italiana
16:15
15m
Talk
Log Parsing with Prompt-based Few-shot Learning
Technical Track
Van-Hoang Le The University of Newcastle, Hongyu Zhang The University of Newcastle
Pre-print
16:30
15m
Talk
Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Technical Track
Noor Nashid University of British Columbia, Mifta Sintaha University of British Columbia, Ali Mesbah University of British Columbia (UBC)
Pre-print
16:45
15m
Paper
An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry
Technical Track
Wenxin Jiang Purdue University, Nicholas Synovic Loyola University Chicago, Matt Hyatt Loyola University Chicago, Taylor R. Schorlemmer Purdue University, Rohan Sethi Loyola University Chicago, Yung-Hsiang Lu Purdue University, George K. Thiruvathukal Loyola University Chicago and Argonne National Laboratory, James C. Davis Purdue University
Pre-print
17:00
15m
Talk
ContraBERT: Enhancing Code Pre-trained Models via Contrastive Learning
Technical Track
Shangqing Liu Nanyang Technological University, bozhi wu Nanyang Technological University, Xiaofei Xie Singapore Management University, Guozhu Meng Institute of Information Engineering, Chinese Academy of Sciences, Yang Liu Nanyang Technological University