Write a Blog >>
ICSE 2023
Sun 14 - Sat 20 May 2023 Melbourne, Australia

How do we know a generated patch is correct? This is a key challenging question that automated program repair (APR) systems struggle to address given the incompleteness of available test suites. Our intuition is that we can triage correct patches by checking whether each generated patch implements code changes (i.e., behaviour) that are relevant to the bug it addresses. Such a bug is commonly specified by a failing test case. Towards predicting patch correctness in APR, we propose a novel yet simple hypothesis on how the link between the patch behaviour and failing test specifications can be drawn: similar failing test cases should require similar patches. We then propose BATS, an unsupervised learning-based approach to predict patch correctness by checking patch Behaviour Against failing Test Specification. BATS exploits deep representation learning models for code and patches: for a given failing test case, the yielded embedding is used to compute similarity metrics in the search for historical similar test cases to identify the associated applied patches, which are then used as a proxy for assessing the correctness of the APR-generated patches. Experimentally, we first validate our hypothesis by assessing whether ground-truth developer patches cluster together in the same way that their associated failing test cases are clustered. Then, after collecting a large dataset of 1,278 plausible patches (written by developers or generated by 32 APR tools), we use BATS to predict correct patches: BATS achieves AUC between 0.557 to 0.718 and recall between 0.562 and 0.854 in identifying correct patches. Our approach outperforms state-of-the-art techniques for identifying correct patches without the need for large labeled patch datasets; as is the case with machine learning-based approaches. While BATS is constrained by the availability of similar test cases, we show that it can still be complementary to existing approaches: when combined with a recent approach that relies on supervised learning, BATS improves the overall recall in detecting correct patches. We finally show that BATS is complementary to the state-of-the-art PATCH-SIM dynamic approach for identifying correct patches generated by APR tools.

Thu 18 May

Displayed time zone: Hobart change

11:00 - 12:30
Program repair techniques and applicationsTechnical Track / Journal-First Papers / DEMO - Demonstrations at Meeting Room 104
Chair(s): Xuan-Bach D. Le University of Melbourne
11:00
15m
Talk
Better Automatic Program Repair by Using Bug Reports and Tests Together
Technical Track
Manish Motwani Georgia Institute of Technology, Yuriy Brun University of Massachusetts
Pre-print
11:15
15m
Talk
CCTEST: Testing and Repairing Code Completion Systems
Technical Track
Li Zongjie , Chaozheng Wang Harbin Institute of Technology, Zhibo Liu Hong Kong University of Science and Technology, Haoxuan Wang EPFL, Dong Chen HKUST, Shuai Wang Hong Kong University of Science and Technology, Cuiyun Gao Harbin Institute of Technology
11:30
7m
Talk
A Controlled Experiment of Different Code Representations for Learning-Based Program Repair
Journal-First Papers
Marjane Namavar University of British Columbia, Noor Nashid University of British Columbia, Ali Mesbah University of British Columbia (UBC)
Link to publication Pre-print
11:37
7m
Talk
Patching Locking Bugs Statically with Crayons
Journal-First Papers
Juan Alfredo Cruz-Carlon IT University of Copenhagen, Mahsa Varshosaz IT University of Copenhagen, Denmark, Claire Le Goues Carnegie Mellon University, Andrzej Wąsowski IT University of Copenhagen, Denmark
11:45
15m
Talk
KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair
Technical Track
Nan Jiang Purdue University, Thibaud Lutellier University of Alberta, Yiling Lou Fudan University, Lin Tan Purdue University, Dan Goldwasser Purdue University, Xiangyu Zhang Purdue University
Pre-print
12:00
15m
Talk
Rete: Learning Namespace Representation for Program RepairDistinguished Paper Award
Technical Track
Nikhil Parasaram University College London, Earl T. Barr University College London, Sergey Mechtaev University College London
Link to publication Pre-print
12:15
7m
Talk
Cerberus: a Program Repair Framework
DEMO - Demonstrations
Ridwan Salihin Shariffdeen National University of Singapore, Martin Mirchev National University of Singapore, Yannic Noller National University of Singapore, Abhik Roychoudhury National University of Singapore
12:22
7m
Talk
Predicting Patch Correctness Based on the Similarity of Failing Test Cases
Journal-First Papers
Haoye Tian University of Luxembourg, Yinghua LI University of Luxembourg, Weiguo Pian University of Luxembourg, Abdoul Kader Kaboré SnT, University of Luxembourg, Kui Liu Huawei Software Engineering Application Technology Lab, Andrew Habib SnT, University of Luxembourg, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé SnT, University of Luxembourg