“An Analysis of Patch Plausibility and Correctness for Generate-and-Validate Patch Generation Systems” critically examines the limits of test‐suite–driven repair by showing that many automatically generated patches, though they pass existing tests, are either semantically incorrect or equivalent to trivial deletions. By introducing Kali—a tool that produces only deletions—this work demonstrates that plausible patches may offer no real repair value, and that relying solely on test outcomes can mask fundamental flaws. Through a systematic re‐evaluation of GenProg, RSRepair, and AE benchmarks, our paper quantifies the gap between patch plausibility and true correctness, advocating for richer evaluation metrics and more robust validation techniques.

In this talk, I will share how an ICSE‐bound paper—intended to introduce a new repair algorithm—was redirected by reviewer feedback into a focused ISSTA study on plausibility versus correctness, reflect on our key results (e.g., Kali outperforming advanced systems and reshaping evaluation norms), and distill lessons for future software engineering research, highlighting the value of critical self‐assessment, negative results, and rigorous evaluation in driving lasting impact.

Bio: Fan Long is an associate professor at Computer Science Department in University of Toronto. He holds PhD of Computer Science from MIT. His research interests include programming language, software engineering, security, and blockchain. He is a recipient of ACM SIGSOFT outstanding dissertation award, ISSTA most influential paper award, and ICSE best paper award. He is also the founder of Conflux, a high-performance next-generation public blockchain project.