Automated Repair of Ambiguous Problem Descriptions for LLM-Based Code Generation
This program is tentative and subject to change.
The widespread adoption of large language models (LLMs) in software engineering has amplified the role of natural language (NL). However, the inherent ambiguity of NL threatens software quality, because ambiguous requirements may lead to faulty program generation. The complexity of ambiguity detection and resolution motivates us to introduce the problem of automated repair of ambiguous NL requirements, which we approach by reducing code generation uncertainty and aligning NL with input-output examples.
Repairing ambiguity in requirements is a difficult challenge for LLMs, as it demands metacognition — the model must understand how its own interpretation changes when the text is altered. Our experiments show that directly prompting an LLM to detect and resolve ambiguities results in irrelevant or inconsistent clarifications. The key novelty we propose is a method of decomposing this problem into simpler sub-problems that do not require metacognitive reasoning. First, we analyze and repair the LLM’s interpretation of requirements embodied by the distribution of programs they induce using traditional testing and program repair methods. Second, we repair requirements based on the changes to the distribution via what we refer to as contrastive specification inference. This decomposition enables targeted, minimal requirement repairs that yield cross-model performance gains in code generation.
This approach is implemented as the tool SpecFix, and evaluated using three state‐of‐the‐art LLMs, GPT-4o, DeepSeek-V3 and Qwen2.5-Coder-32b-Instruct, across two widely used code generation benchmarks: HumanEval+ and MBPP+. Our results show that SpecFix, operating autonomously without human intervention or external information, modifies 23.93% of the requirements, leading to a 33.66% improvement in model Pass@1 on the modified requirements. Across the entire benchmark, this corresponds to an absolute increase of 4.3% in overall Pass@1.
This program is tentative and subject to change.
Wed 19 NovDisplayed time zone: Seoul change
11:00 - 12:30 | |||
11:00 10mTalk | Automated Repair of Ambiguous Problem Descriptions for LLM-Based Code Generation Research Papers Haoxiang Jia Peking University, Robbie Morris University College London, He Ye University College London (UCL), Federica Sarro University College London, Sergey Mechtaev Peking University | ||
11:10 10mTalk | Fixing Broken Graphs: LLM-Powered Automatic Code Optimization for DNN Programs Research Papers Haotian Wang Nankai University, Yicheng Sui Nankai University, Yudong Xie Nankai University, Yicong Liu Nankai University, Yufei Sun Nankai University, Changqing Shi Nankai University, Yuzhi Zhang Nankai University | ||
11:20 10mTalk | SemGuard: Real-Time Semantic Evaluator for Correcting LLM-Generated Code Research Papers Qinglin Wang Shandong Normal University, Zhihong Sun Shandong Normal University, Ruyun Wang Institute of Information Engineering, Chinese Academy of Sciences, Tao Huang Shandong Normal University, Zhi Jin Peking University, Ge Li Peking University, Chen Lyu Shandong Normal University | ||
11:30 10mTalk | Amur: Fixing Multi-Resource Leaks Guided by Resource Flow Analysis Research Papers | ||
11:40 10mTalk | Automated Repair of OpenID Connect Programs Research Papers Tamjid Al Rahat University of Virginia, Yanju Chen University of California, San Diego, Yu Feng University of California at Santa Barbara, Yuan Tian | ||
11:50 10mTalk | FlakyGuard: Automatically Fixing Flaky Tests at Industry Scale Research Papers Chengpeng Li University of Texas at Austin, Farnaz Behrang Uber Technologies, August Shi The University of Texas at Austin, Peng Liu Uber Technologies | ||
12:00 10mTalk | LLMPort: Cross-file Patch Porting via Task Decomposition and Self-correction Research Papers Bofei Chen Fudan University, Lei Zhang Fudan University, Peng Deng Fudan University, Nan Wang Fudan University, Haoyu Xu Fudan University, Mingda Guo Fudan Universityv, Yuan Zhang Fudan University, Min Yang Fudan University | ||
12:10 10mTalk | Repairing Leaks in Resource Wrappers Research Papers Sanjay Malakar University of California, Riverside, Martin Kellogg New Jersey Institute of Technology, Michael D. Ernst University of Washington, Manu Sridharan University of California at Riverside | ||
12:20 10mTalk | Automatic Fixing of Missing Dependency Errors Research Papers Jun Lyu Nanjing University, He Zhang Nanjing University, Lanxin Yang Nanjing University, Yue Li Nanjing University, Chenxing Zhong Nanjing University, Manuel Rigger National University of Singapore | ||