Toward Repository-Level Program Verification with Large Language Models
Recent advancements in large language models (LLMs) suggest great promises in code and proof generations. However, scaling automated formal verification to real-world projects requires resolving cross-module dependencies and global contexts, which are crucial challenges overlooked by existing LLM-based methods with a special focus on targeting isolated, function-level verification tasks. To systematically explore and address the significant challenges of verifying entire software repositories, we introduce RVBench, the first verification benchmark explicitly designed for repository-level evaluation, constructed from four diverse and complex open-source Verus projects.
We further introduce RagVerus, an extensible framework that synergizes retrieval-augmented generation with context-aware prompting to automate proof synthesis for multi-module repositories. RagVerus triples proof pass rates on existing benchmarks under constrained model inference budgets, and achieves a 27% relative improvement on the more challenging RVBench benchmark, demonstrating a scalable and sample-efficient verification solution.
Wed 15 OctDisplayed time zone: Perth change
| 16:00 - 17:40 | LLMs for Program Analysis and Verification IILMPL at Orchid East Chair(s): Zhuo Zhang Columbia University | ||
| 16:0015m Talk | Hallucination-Resilient LLM-Driven Sound and Tunable Static Analysis LMPL | ||
| 16:1520m Talk | Toward Repository-Level Program Verification with Large Language Models LMPLDOI Pre-print | ||
| 16:3515m Talk | Preguss: It Analyzes, It Specifies, It Verifies LMPL Zhongyi Wang Zhejiang University, China, Tengjie Lin Zhejiang University, Mingshuai Chen Zhejiang University, Mingqi Yang Zhejiang University, Haokun Li Peking University, Xiao Yi The Chinese University of Hong Kong, Shengchao Qin Xidian University, Jianwei Yin Zhejiang University | ||
| 16:5020m Talk | A Case Study on the Effectiveness of LLMs in Verification with Proof Assistants LMPL Barış Bayazıt University of Toronto, Yao Li Portland State University, Xujie Si University of TorontoDOI Pre-print | ||
| 17:1020m Talk | Understanding Formal Reasoning Failures in LLMs as Abstract Interpretersremote LMPL Jacqueline Mitchell University of Southern California, Brian Hyeongseok Kim University of Southern California, Chenyu Zhou University of Southern California, Chao Wang University of Southern California | ||

