utomated Program Repair (APR) can assist developers by automatically generating patches for buggy code. However, as recent techniques leverage deep learning models, developers do not know why the model generated a particular patch. Existing Explainable AI (XAI) techniques, such as SHAP, can be applied to APR, however, their complexity raises questions about whether developers find such explanations understandable. In this work, we develop a novel framework SCHOLIA, with two extensions to feature attribution methods to make them more understandable to the developers. First generates a text explanation based on attribution scores. Second creates a visualization capturing the transformation of the patch based on the impact of code tokens, named patch transformation.
We evaluated the proposed new two explanations types compared to SHAP, using a user survey. The survey received responses from 106 participants. Accordingly, 68.9% (P $<$ .05) and 64.2% (P $<$ .05) of participants agreed that text explanation and patch transformation methods are easy to understand, while only 17.9% (P $<$ .05) agreed with the original SHAP explanation. The survey responses indicate, with statistical significance, that our extensions to SHAP are easier to understand than the original SHAP explanations.
Tue 29 AprDisplayed time zone: Eastern Time (US & Canada) change
14:00 - 15:30 | APR Session 3APR at 210 Chair(s): Tegawendé F. Bissyandé University of Luxembourg, Chao Peng ByteDance | ||
14:00 30mOther | Discussion APR Chao Peng ByteDance | ||
14:30 20mTalk | Bogus Bugs, Duplicates, and Revealing Comments: Data Quality Issues in NPR APR | ||
14:50 20mTalk | LLM-Based Repair of C++ Implicit Data Loss Compiler Warnings: An Industrial Case Study APR | ||
15:10 20mTalk | Scholia - An XAI Framework for APR APR Nethum Lamahewage University of Moratuwa, Sri Lanka, Nimantha Cooray University of Moratuwa, Sri Lanka, Ridwan Salihin Shariffdeen National University of Singapore, Sandareka Wickramanayake University of Moratuwa, Sri Lanka, Nisansa de Silva University of Moratuwa, Sri Lanka |