Towards Better Software Quality in the Era of Large Language Models
Abstract: Large Language Models (LLMs), such as ChatGPT, have shown impressive performance in various downstream tasks spanning diverse fields. In this talk, I will present our recent work on leveraging LLMs for improving software quality, covering techniques for breaking, fixing, and synthesizing software systems. More specifically, I will first talk about our TitanFuzz work, the first approach demonstrating that LLMs can be directly applied for both generation- and mutation-based fuzz testing studied for decades, while being fully automated, generalizable, and applicable to challenging application domains (such as ML systems). Next, I will talk about our AlphaRepair work, which reformulates the Automated Program Repair (APR) problem as an infilling (or cloze) task and demonstrates that LLMs can outperform all prior APR techniques studied for over a decade. Lastly, I will briefly talk about our recent EvalPlus work, which shows that the evaluation of almost all recent LLMs on program synthesis can be largely affected by the weak test suites in existing datasets. Furthermore, I will also briefly talk about our other work along the covered directions.
Lingming Zhang is an Associate Professor at the Department of Computer Science in University of Illinois Urbana-Champaign. His main research interests lie in Software Engineering, and its synergy with Machine Learning, Programming Languages, and Formal Methods. He has published over 80 research papers, winning the ACM SIGSOFT Early Career Researcher Award, four ACM SIGSOFT Distinguished Paper Awards, and one Best Industry Paper Award. His research has helped detect hundreds of bugs for open-source projects from Apache and GitHub, as well as software systems from eBay, eMetric, Google, Meta/Facebook, Microsoft, NVIDIA, OctoML, Oracle, and Yahoo!. His work on regression testing optimization has been run day-to-day in Google, while his work on automated program repair and unified debugging has been successfully deployed to the Alipay system with million lines of code and over 1 billion global users.
Thu 14 SepDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
13:20 - 15:20 | SATE - Software Engineering at the Era of LLMsSATE - Software Engineering at the Era of LLMs at Room FR Chair(s): Xin Xia Huawei Technologies | ||
13:20 40mTalk | Towards Better Software Quality in the Era of Large Language Models SATE - Software Engineering at the Era of LLMs Lingming Zhang University of Illinois at Urbana-Champaign | ||
14:00 40mTalk | Securing LLM-based Software Supply Chains SATE - Software Engineering at the Era of LLMs Audris Mockus Vilnius University & The University of Tennessee File Attached | ||
14:40 40mTalk | BEWARE: some of the deep learning rhetoric is misleading SATE - Software Engineering at the Era of LLMs Tim Menzies North Carolina State University Pre-print | ||
