Reflective Unit Test Generation for Precise Type Error Detection with Large Language Models
This program is tentative and subject to change.
Type errors in Python often lead to runtime failures, posing significant challenges to software reliability and developer productivity. Existing static analysis tools aim to detect such errors without execution but frequently suffer from high false positive rates. Recently, unit test generation techniques offer great promise in achieving high test coverage, but they often struggle to produce bug-revealing tests without tailored guidance. To address these limitations, we present RTED, a novel type-aware test generation technique for automatically detecting Python type errors. Specifically, RTED combines step-by-step type constraint analysis with reflective validation to guide the test generation process and effectively suppress false positives. We evaluated RTED on two widely-used benchmarks, BugsInPy and TypeBugs. Experimental results show that RTED can detect 22$\sim$29 more benchmarked type errors than four state-of-the-art techniques. RTED is also capable of producing fewer false positives, achieving an improvement of 173.9% $\sim$ 245.9% in precision. Furthermore, we applied RTED to six real-world open-source Python projects, and successfully discovered 12 previously unknown type errors, demonstrating RTED’s practical value.
This program is tentative and subject to change.
Wed 19 NovDisplayed time zone: Seoul change
11:00 - 12:30 | |||
11:00 10mTalk | PALM: Synergizing Program Analysis and LLMs to Enhance Rust Unit Test Coverage Research Papers | ||
11:10 10mTalk | ROR-DSE: ROR adequate test case generation using dynamic symbolic execution Journal-First Track Sangharatna Godboley NIT Warangal | ||
11:20 10mTalk | Reflective Unit Test Generation for Precise Type Error Detection with Large Language Models Research Papers Chen Yang Tianjin University, Ziqi Wang Tianjin University, Yanjie Jiang Peking University, Lin Yang Tianjin University, Yuteng Zheng Tianjin University, Jianyi Zhou Huawei Cloud Computing Technologies Co., Ltd., Junjie Chen Tianjin University | ||
11:30 10mTalk | FailMapper: Automated Generation of Unit Tests Guided by Failure Scenarios Research Papers ruiqi dong Swinburne University of Technology, Zehang Deng Swinburne University of Technology, Xiaogang Zhu The University of Adelaide, Xiaoning Du Monash University, Huai Liu Swinburne University of Technology, Shaohua Wang Central University of Finance and Economics, Sheng Wen Swinburne University of Technology, Yang Xiang Digital Research & Innovation Capability Platform, Swinburne University of Technology | ||
11:40 10mTalk | Advancing Code Coverage: Incorporating Program Analysis with Large Language Models Journal-First Track Chen Yang Tianjin University, Junjie Chen Tianjin University, Bin Lin Hangzhou Dianzi University, Ziqi Wang Tianjin University, Jianyi Zhou Huawei Cloud Computing Technologies Co., Ltd. | ||
11:50 10mTalk | Navigating the Labyrinth: Path-Sensitive Unit Test Generation with Large Language Models Research Papers Dianshu Liao the Australian National University, Xin Yin Zhejiang University, Shidong Pan Columbia University & New York University, Chao Ni Zhejiang University, Zhenchang Xing CSIRO's Data61, Xiaoyu Sun Australian National University, Australia Pre-print | ||
12:00 10mTalk | Enhancing LLM’s Ability to Generate More Repository-Aware Unit Tests Through Precise Context Injection Research Papers Xin Yin Zhejiang University, Chao Ni Zhejiang University, Xinrui Li School of Software Technology, Zhejiang University, Liushan Chen Douyin Co., Ltd., Guojun Ma Douyin Co., Ltd., Xiaohu Yang Zhejiang University Pre-print | ||
12:10 10mTalk | Toward Cost-Effective Adaptive Random Testing: An Approximate Nearest Neighbor Approach Journal-First Track Rubing Huang Macau University of Science and Technology (M.U.S.T.), Chenhui Cui Macau University of Science and Technology, Junlong Lian Jiangsu University, Haibo Chen Jiangsu University, Dave Towey University of Nottingham Ningbo China, Weifeng Sun | ||
12:20 10mTalk | Automated Combinatorial Test Generation for Alloy Research Papers Agustín Borda Dept. of Computer Science FCEFQyN, University of Rio Cuarto, Germán Regis University of Rio Cuarto and CONICET, Nazareno Aguirre University of Rio Cuarto and CONICET, Marcelo F. Frias Dept. of Software Engineering Instituto Tecnológico de Buenos Aires, Pablo Ponzio Dept. of Computer Science FCEFQyN, University of Rio Cuarto | ||