Multifaceted Hierarchical Report Identification for Non-Functional Bugs in Deep Learning Frameworks
Non-functional bugs (e.g., performance- or accuracy-related bugs) in Deep Learning (DL) frameworks can lead to some of the most devastating consequences. Reporting those bugs on a repository such as GitHub is a standard route to fix them. Yet, given the growing number of new GitHub reports for DL frameworks, it is intrinsically difficult for developers to distinguish those that reveal non-functional bugs among the others, and assign them to the right contributor for investigation in a timely manner. In this paper, we propose MHNurf Ean end-to-end tool for automatically identifying non-functional bug related reports in DL frameworks. The core of MHNurf is a Multifaceted Hierarchical Attention Network (MHAN) that tackles three unaddressed challenges: (1) learning the semantic knowledge, but doing so by (2) considering the hierarchy (e.g., words/tokens in sentences/statements) and focusing on the important parts (i.e., words, tokens, sentences, and statements) of a GitHub report, while (3) independently extracting information from different types of features, i.e., content, comment, code, command, and label. To evaluate MHNurf, we leverage 3,721 GitHub reports from five DL frameworks for conducting experiments. The results show that MHNurf works the best with a combination of content, comment, and code, which considerably outperforms the classic HAN where only the content is used. MHNurf also produces significantly more accurate results than nine other state-of-theart classifiers with strong statistical significance, i.e., up to 71% AUC improvement and has the best Scott-Knott rank on four frameworks while 2nd on the remaining one. To facilitate reproduction and promote future research, we have made our dataset, code, and detailed supplementary results publicly available at: https://github.com/anonymoususr12/MHNurf.
Thu 8 DecDisplayed time zone: Osaka, Sapporo, Tokyo change
13:00 - 14:30 | |||
13:00 20mPaper | Incremental Relational Topic Model for Duplicate Bug Report Detection Technical Track | ||
13:20 20mPaper | Multifaceted Hierarchical Report Identification for Non-Functional Bugs in Deep Learning Frameworks Technical Track Guoming Long Loughborough University, Tao Chen Loughborough University, Georgina Cosma Loughborough University | ||
13:40 20mPaper | Automatic Generation of Metamorphic Relations for a Cyber-Physical System-of-Systems Using Genetic Algorithm Technical Track Esther Cho Korea Advanced Institute of Science and Technology, Yong-Jun Shin Korea Advanced Institute of Science and Technology (KAIST), Sangwon Hyun Korea Advanced Institute of Science and Technology, Hansu Kim Korea Advanced Institute of Science and Technology, Doo-Hwan Bae Korea Advanced Institute of Science and Technology | ||
14:00 20mPaper | An approach for hazard analysis of multiple-cooperative systems considering dynamic configuration uncertainty Technical Track |