TCSE logo 
 Sigsoft logo
Sustainability badge
Wed 30 Apr 2025 12:15 - 12:30 at 212 - AI for Analysis 1 Chair(s): Denys Poshyvanyk

Formal verification using proof assistants, such as Coq, allows for high-quality software. However, the verification process is expensive, requiring significant expertise and manual effort to write proofs. Recent work has explored automating proof synthesis using machine learning, and even more recently, large language models (LLMs), showing that retrieving relevant premises (such as lemmas and definitions) is helpful for these models. We present Rango, a fully automated proof synthesis tool for Coq that uses, not only relevant premises but also similar proofs from the current project. Rango uses retrieval augmentation at every step of the proof to automatically determine which proofs and premises to include in the context of its fine-tuned LLM. In this way, Rango adapts to the project and to the evolving state of the proof. We create a new dataset, CoqStoq, of 2,205 open-source Coq projects from GitHub, which includes both training data and a curated evaluation benchmark of well-maintained projects. On this benchmark, Rango synthesizes 27.7% of the proofs, which is 10% more proofs than prior state-of-the-art tool Tactician. Our evaluation also shows that adding relevant proofs to the context in Rango leads to a 45% increase in the number of theorems proven.

Slides (icse-2025.pptx)6.60MiB

Wed 30 Apr

Displayed time zone: Eastern Time (US & Canada) change

11:00 - 12:30
AI for Analysis 1Research Track at 212
Chair(s): Denys Poshyvanyk William & Mary
11:00
15m
Talk
A Multiple Representation Transformer with Optimized Abstract Syntax Tree for Efficient Code Clone Detection
Research Track
TianChen Yu School of Software Engineering, South China University of Technology, Li Yuan School of Software Engineering, South China University of Technology, Guangzhou, China, Liannan Lin School of Software Engineering, South China University of Technology, Hongkui He School of Software Engineering, South China University of Technology
11:15
15m
Talk
Can an LLM find its way around a Spreadsheet?
Research Track
Cho-Ting Lee Virginia Tech, Andrew Neeser Virginia Tech, Shengzhe Xu Virginia Tech, Jay Katyan Virginia Tech, Patrick Cross Virginia Tech, Sharanya Pathakota Virginia Tech, Marigold Norman World Forest ID, John C. Simeone Simeone Consulting, LLC, Jaganmohan Chandrasekaran Virginia Tech, Naren Ramakrishnan Virginia Tech
11:30
15m
Talk
QEDCartographer: Automating Formal Verification Using Reward-Free Reinforcement LearningArtifact-Available
Research Track
Alex Sanchez-Stern University of Massachusetts at Amherst, Abhishek Varghese University of Massachusetts, Zhanna Kaufman University of Massachusetts, Shizhuo Zhang University of Illinois Urbana-Champaign, Talia Lily Ringer University of Illinois Urbana-Champaign, Yuriy Brun University of Massachusetts
Link to publication Pre-print
11:45
15m
Talk
TIGER: A Generating-Then-Ranking Framework for Practical Python Type Inference
Research Track
Chong Wang Nanyang Technological University, Jian Zhang Nanyang Technological University, Yiling Lou Fudan University, Mingwei Liu Fudan University, Weisong Sun Nanyang Technological University, Yang Liu Nanyang Technological University, Xin Peng Fudan University
12:00
15m
Talk
ROCODE: Integrating Backtracking Mechanism and Program Analysis in Large Language Models for Code Generation
Research Track
Xue Jiang , Yihong Dong Peking University, Yongding Tao University of Electronic Science and Technology of China, Huanyu Liu Xidian University, Zhi Jin Peking University, Ge Li Peking University
12:15
15m
Talk
Rango: Adaptive Retrieval-Augmented Proving for Automated Software VerificationArtifact-FunctionalArtifact-AvailableArtifact-ReusableAward Winner
Research Track
Kyle Thompson University of California, San Diego, Nuno Saavedra INESC-ID and IST, University of Lisbon, Pedro Carrott Imperial College London, Kevin Fisher University of California San Diego, Alex Sanchez-Stern University of Massachusetts, Yuriy Brun University of Massachusetts, João F. Ferreira INESC-ID and IST, University of Lisbon, Sorin Lerner University of California at San Diego, Emily First University of California, San Diego
Link to publication Pre-print File Attached
:
:
:
: