Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors
Large language models (LLMs) have been adopted for text-to-SQL tasks, utilizing their in-context learning (ICL) capability to translate natural language questions into SQL queries. However, such a technique faces correctness problems. In this paper, we conduct the first comprehensive study of text-to-SQL errors of ICL-based techniques. Our study covers four representative ICL-based techniques, five basic repairing methods, two benchmarks, and two LLM settings. We find that text-to-SQL errors are widespread and summarize 27 error types of 7 categories. We also find that existing repairing attempts have limited correctness improvement while having high computational overhead and many mis-repairs. Based on these findings, we propose MapleDoctor, a novel text-to-SQL error detection and repairing framework. The evaluation demonstrates that MapleDoctor outperforms existing solutions by repairing 13.8% more queries with neglectable mis-repairs and reducing 67.4% overhead. The artifact is publicly available at GitHub.
Tue 7 JulDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | Program RepairIdeas, Visions and Reflections / Research Papers / Industry Papers at MB 3.270 Chair(s): Lwin Khin Shar Singapore Management University | ||
11:00 20mTalk | Automated Repair of TEE Partitioning Issues via DSL-Guided and LLM-Assisted Patching Research Papers Chengyan Ma Singapore Management University, Jieke Shi Singapore Management University, Ruidong Han Singapore Management University, Ye Liu Singapore Management University, FENG Li , Yuqing Niu , David Lo Singapore Management University Pre-print | ||
11:20 20mTalk | TLR: Codebase-Level C Memory Management Error Repair with Large Language Models Research Papers Xiao Cheng Macquarie University, Zhihao Guo UTS, Huan Huo University of Technology Sydney, Yulei Sui University of New South Wales Pre-print | ||
11:40 20mTalk | AutoCodeRover: Agentic Program Repair for SonarQube Issues Industry Papers Martin Mirchev National University of Singapore, Ridwan Salihin Shariffdeen SonarSource, Haifeng Ruan National University of Singapore, Yuntong Zhang National University of Singapore, Abhik Roychoudhury National University of Singapore DOI Pre-print | ||
12:00 10mTalk | Who Wrote This Patch? Toward Accountable Automated Program Repair Ideas, Visions and Reflections Huaijin Ran Xi’an Jiaotong-Liverpool University, Haoyi Zhang Xi’an Jiaotong-Liverpool University, Kisub Kim DGIST, Xunzhu Tang University of Luxembourg DOI | ||
12:10 20mTalk | Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors Research Papers Jiawei Shen East China Normal University, Chengcheng Wan East China Normal University, Ruoyi Qiao East China Normal University, Jiazhen Zou East China Normal University, Hang Xu East China Normal University, Yuchen Shao East China Normal University, Shanghai Innovation Institute, Yueling Zhang East China Normal University, Weikai Miao East China Normal University, Geguang Pu East China Normal University, China Pre-print | ||