Combining Language and App UI Analysis for the Automated Assessment of Bug Reproduction Steps
Bug reports are essential for developers to confirm software problems, investigate their causes, and validate fixes. Unfortunately, reports often miss important information or are written unclearly, which can cause delays, extra issue resolution efforts, or even the inability to solve the issues. One of the often problematic components of reports is the steps to reproduce the bugs (S2Rs), which are essential to replicate the described bug(s) and reason about fixes. Given the proclivity for deficiencies in reported S2Rs, prior work has proposed techniques that assist reporters in writing S2Rs or assessing the quality of reported S2Rs. However, automated understanding of S2Rs is a challenging problem that requires linking nuanced natural language phrases with specific program information – which prior techniques struggle with due to issues related to language variability and limitations of information gleaned from program analysis. Such challenges make it difficult to infer the language <–> program connections needed for S2R understanding.
To more effectively tackle the problem of S2R quality annotation, we propose a new technique called AstroBR which leverages the language understanding capabilities of LLMs to identify and extract, the S2Rs from bug reports and match them to GUI interactions in a program state model gleaned from dynamic analysis. We compared AstroBR to a related state-of-the-art approach and we found that AstroBR annotates S2Rs 25.2% better (in terms of F1 score) than the baseline. Additionally, AstroBR suggests more accurate missing S2Rs than the baseline (by 71.4% in terms of F1 score.).
Sun 27 AprDisplayed time zone: Eastern Time (US & Canada) change
09:00 - 10:30 | App ComprehensionResearch Track / / at 205 Chair(s): Coen De Roover Vrije Universiteit Brussel, Gema Rodríguez-Pérez Department of Computer Science, Mathematics, Physics and Statistics, University of British Columbia, Okanagan Campus | ||
09:00 15mDay opening | Welcome by the Chairs Opening | ||
09:15 50mKeynote | Theories of Program Comprehension in the Age of LLMs ICPC Keynotes | ||
10:05 10mTalk | Combining Language and App UI Analysis for the Automated Assessment of Bug Reproduction Steps Research Track Junayed Mahmud University of Central Florida, Antu Saha William & Mary, Oscar Chaparro William & Mary, Kevin Moran University of Central Florida, Andrian Marcus George Mason University Pre-print | ||
10:15 10mTalk | Characterizing Bugs in Login Processes of Android Applications: An Empirical Study Research Track Zixu Zhou McGill University, Rufeng Chen McGill University, Junfeng Chen Southern University of Science and Technology, Yepang Liu Southern University of Science and Technology, Lili Wei McGill University Pre-print | ||
10:25 5mLive Q&A | Session's Discussion: "App Comprehension" Research Track |