Out of Context: How important is Local Context in Neural Program Repair?
Deep learning source code models have been applied very successfully to the problem of automated program repair. One of the standing issues is the small input window of current models which often cannot fully fit the context code required for a bug fix (e.g., method or class declarations of a project). Instead, input is often restricted to the local context, that is, the lines below and above the bug location. In this work we study the importance of this local context on repair success: how much local context is needed?; is context before or after the bug location more important? how is local context tied to the bug type? To answer these questions we train and evaluate Transformer models in many different local context configurations on three datasets and two programming languages. Our results indicate that overall repair success increases with the size of the local context (albeit not for all bug types) and confirm the common practice that roughly 50-60% of the input window should be used for context leading the bug. Our results are not only relevant for researchers working on Transformer-based APR tools but also for benchmark and dataset creators who must decide what and how much context to include in their datasets.
Wed 17 AprDisplayed time zone: Lisbon change
16:00 - 17:30 | |||
16:00 15mTalk | RUNNER: Responsible UNfair NEuron Repair for Enhancing Deep Neural Network Fairness Research Track Li Tianlin Nanyang Technological University, Yue Cao Nanyang Technological University, Jian Zhang Nanyang Technological University, Shiqian Zhao Nanyang Technological University, Yihao Huang East China Normal University, Aishan Liu Beihang University; Institute of Dataspace, Qing Guo IHPC and CFAR at A*STAR, Singapore, Yang Liu Nanyang Technological University | ||
16:15 15mTalk | ITER: Iterative Neural Repair for Multi-Location Patches Research Track | ||
16:30 15mTalk | Out of Context: How important is Local Context in Neural Program Repair? Research Track | ||
16:45 15mTalk | Out of Sight, Out of Mind: Better Automatic Vulnerability Repair by Broadening Input Ranges and Sources Research Track Xin Zhou Singapore Management University, Singapore, Kisub Kim Singapore Management University, Singapore, Bowen Xu North Carolina State University, DongGyun Han Royal Holloway, University of London, David Lo Singapore Management University | ||
17:00 15mTalk | Strengthening Supply Chain Security with Fine-grained Safe Patch Identification Research Track Luo Changhua The Chinese University of Hong Kong, Wei Meng Chinese University of Hong Kong, Shuai Wang The Hong Kong University of Science and Technology |