Human-in-the-Loop Oracle Learning for Semantic Bugs in String Processing Programs
Thu 21 Jul 2022 17:20 - 17:40 at ISSTA 2 - Session 3-6: Neural Networks, Learning, NLP F
How can we automatically repair semantic bugs in string-processing programs? A semantic bug is an unexpected program state: The program does not crash (which can be easily detected). Instead, the program processes the input incorrectly. It produces an output which users identify as unexpected. We envision a fully automated debugging process for semantic bugs where a user reports the unexpected behavior for a given input and the machine negotiates the condition under which the program fails. During the negotiation, the machine learns to predict the user’s response and in this process learns an automated oracle for semantic bugs.
In this paper, we introduce Grammar2Fix, an automated oracle learning and debugging technique for string-processing programs even when the input format is unknown. Grammar2Fix represents the oracle as a regular grammar which is iteratively improved by systematic queries to the user for other inputs that are likely failing. Grammar2Fix implements several heuristics to maximize the oracle quality under a minimal query budget. In our experiments with three widely-used repair benchmark sets, predicts passing inputs as passing and failing inputs as failing with more than 96% precision and recall, using a median of 42 queries to the user.
Wed 20 JulDisplayed time zone: Seoul change
07:00 - 08:20 | |||
07:00 20mTalk | Cross-Lingual Transfer Learning for Statistical Type InferenceACM SIGSOFT Distinguished Paper Technical Papers Zhiming Li Nanyang Technological University, Singapore, Xiaofei Xie Singapore Management University, Singapore, Haoliang Li City University of Hong Kong, Zhengzi Xu Nanyang Technological University, Yi Li Nanyang Technological University, Singapore, Yang Liu Nanyang Technological University DOI | ||
07:20 20mTalk | DocTer: Documentation-Guided Fuzzing for Testing Deep Learning API Functions Technical Papers Danning Xie Purdue University, Yitong Li University of Waterloo, Mijung Kim UNIST, Hung Viet Pham University of Waterloo, Lin Tan Purdue University, Xiangyu Zhang Purdue University, Michael W. Godfrey University of Waterloo, Canada DOI | ||
07:40 20mTalk | HybridRepair: Towards Annotation-Efficient Repair for Deep Learning Models Technical Papers DOI | ||
08:00 20mTalk | Human-in-the-Loop Oracle Learning for Semantic Bugs in String Processing Programs Technical Papers Charaka Geethal Monash University, Thuan Pham The University of Melbourne, Aldeida Aleti Monash University, Marcel Böhme MPI-SP, Germany and Monash University, Australia DOI Pre-print | ||
Thu 21 JulDisplayed time zone: Seoul change
16:20 - 17:40 | |||
16:20 20mTalk | AEON: A Method for Automatic Evaluation of NLP Test Cases Technical Papers Jen-tse Huang The Chinese University of Hong Kong, Jianping Zhang The Chinese University of Hong Kong, Wenxuan Wang The Chinese University of Hong Kong, Pinjia He The Chinese University of Hong Kong, Shenzhen, Yuxin Su Sun Yat-sen University, Michael Lyu The Chinese University of Hong Kong DOI | ||
16:40 20mTalk | HybridRepair: Towards Annotation-Efficient Repair for Deep Learning Models Technical Papers DOI | ||
17:00 20mTalk | Improving Cross-Platform Binary Analysis using Representation Learning via Graph Alignment Technical Papers Geunwoo Kim University of California, Irvine, USA, Sanghyun Hong Oregon State University, Michael Franz University of California, Irvine, Dokyung Song Yonsei University, South Korea DOI | ||
17:20 20mTalk | Human-in-the-Loop Oracle Learning for Semantic Bugs in String Processing Programs Technical Papers Charaka Geethal Monash University, Thuan Pham The University of Melbourne, Aldeida Aleti Monash University, Marcel Böhme MPI-SP, Germany and Monash University, Australia DOI Pre-print | ||