Better Automatic Program Repair by Using Bug Reports and Tests Together
Automated program repair is already deployed in industry, but concerns remain about repair quality. Recent research has shown that one of the main reasons repair tools produce incorrect (but seemingly correct) patches is imperfect fault localization (FL). This paper demonstrates that combining information from natural-language bug reports and test executions when localizing bugs can have a significant positive impact on repair quality. By modifying existing repair tools to use FL that combines bug reports and tests, we are able to correctly repair 7 defects in Defects4J that no prior tools have repaired correctly.
We develop, Blues, the first information-retrieval-based FL technique that requires no training data. We further develop RAFL, the first unsupervised method for combining multiple FL techniques, which outperforms an existing supervised method. Using RAFL, we create SBIR by combining Blues with a spectrum-based (SBFL) technique. Evaluated on 815 real-world defects, SBIR consistently ranks buggy statements higher than its underlying techniques.
Finally, we modify three state-of-the-art repair tools, Arja, SequenceR, and SimFix, to use SBIR, SBFL, and Blues as their internal FL. We evaluate the quality of the produced patches on 689 real-world defects. Arja and SequenceR significantly benefit from SBIR: Arja using SBIR correctly repairs 28 defects, but only 21 using SBFL, and only 15 using Blues; SequenceR using SBIR correctly repairs 12 defects, but only 10 using SBFL, and only 4 using Blues. SimFix, (which has internal mechanisms to overcome poor FL), correctly repairs 30 defects using SBIR and SBFL, but only 13 using Blues. Our promising findings direct further research into combining data from bug reports and test executions for FL and program repair.
Thu 18 MayDisplayed time zone: Hobart change
11:00 - 12:30 | Program repair techniques and applicationsTechnical Track / Journal-First Papers / DEMO - Demonstrations at Meeting Room 104 Chair(s): Xuan-Bach D. Le University of Melbourne | ||
11:00 15mTalk | Better Automatic Program Repair by Using Bug Reports and Tests Together Technical Track Pre-print | ||
11:15 15mTalk | CCTEST: Testing and Repairing Code Completion Systems Technical Track Li Zongjie , Chaozheng Wang Harbin Institute of Technology, Zhibo Liu Hong Kong University of Science and Technology, Haoxuan Wang EPFL, Dong Chen HKUST, Shuai Wang Hong Kong University of Science and Technology, Cuiyun Gao Harbin Institute of Technology | ||
11:30 7mTalk | A Controlled Experiment of Different Code Representations for Learning-Based Program Repair Journal-First Papers Marjane Namavar University of British Columbia, Noor Nashid University of British Columbia, Ali Mesbah University of British Columbia (UBC) Link to publication Pre-print | ||
11:37 7mTalk | Patching Locking Bugs Statically with Crayons Journal-First Papers Juan Alfredo Cruz-Carlon IT University of Copenhagen, Mahsa Varshosaz IT University of Copenhagen, Denmark, Claire Le Goues Carnegie Mellon University, Andrzej Wąsowski IT University of Copenhagen, Denmark | ||
11:45 15mTalk | KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair Technical Track Nan Jiang Purdue University, Thibaud Lutellier University of Alberta, Yiling Lou Fudan University, Lin Tan Purdue University, Dan Goldwasser Purdue University, Xiangyu Zhang Purdue University Pre-print | ||
12:00 15mTalk | Rete: Learning Namespace Representation for Program Repair Technical Track Nikhil Parasaram University College London, Earl T. Barr University College London, Sergey Mechtaev University College London Link to publication Pre-print | ||
12:15 7mTalk | Cerberus: a Program Repair Framework DEMO - Demonstrations Ridwan Salihin Shariffdeen National University of Singapore, Martin Mirchev National University of Singapore, Yannic Noller National University of Singapore, Abhik Roychoudhury National University of Singapore | ||
12:22 7mTalk | Predicting Patch Correctness Based on the Similarity of Failing Test Cases Journal-First Papers Haoye Tian University of Luxembourg, Yinghua LI University of Luxembourg, Weiguo Pian University of Luxembourg, Abdoul Kader Kaboré SnT, University of Luxembourg, Kui Liu Huawei Software Engineering Application Technology Lab, Andrew Habib SnT, University of Luxembourg, Jacques Klein University of Luxembourg, Tegawendé F. Bissyandé SnT, University of Luxembourg | ||