ASE 2024
Sun 27 October - Fri 1 November 2024 Sacramento, California, United States
Thu 31 Oct 2024 13:30 - 13:45 at Carr - Code generation 3 Chair(s): Jialun Cao

Recent Large Language Models (LLMs) have demonstrated significant capabilities in generating code snippets directly from problem statements. This increasingly automated process mirrors traditional human-led software development, where code is often written in response to a requirement. Historically, Test-Driven Development (TDD) has proven its merit, requiring developers to write tests before the functional code, ensuring alignment with the initial problem statements. Applying TDD principles to LLM-based code generation offers one distinct benefit: it enables developers to verify the correctness of generated code against predefined tests. This paper investigates if and how TDD can be incorporated into AI-assisted code-generation processes. We experimentally evaluate our hypothesis that providing LLMs like \textit{GPT-4} and \textit{Llama 3} with tests in addition to the problem statements enhances code generation outcomes. We experimented with established function-level code generation benchmarks such as MBPP and HumanEval. Our results consistently demonstrate that including test cases leads to higher success in solving programming challenges. We assert that TDD is a promising paradigm for helping ensure that the code generated by LLMs effectively captures the requirements.

Thu 31 Oct

Displayed time zone: Pacific Time (US & Canada) change

13:30 - 15:00
Code generation 3Industry Showcase / Journal-first Papers / Research Papers at Carr
Chair(s): Jialun Cao Hong Kong University of Science and Technology
13:30
15m
Talk
Test-Driven Development and LLM-based Code Generation
Research Papers
Noble Saji Mathews University of Waterloo, Canada, Mei Nagappan University of Waterloo
13:45
15m
Talk
A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven RefinementACM SigSoft Distinguished Paper Award
Research Papers
Huan Zhang Nanjing University, Wei Cheng Nanjing University, Yuhan Wu Nanjing University, Wei Hu Nanjing University
14:00
15m
Talk
Ansible Lightspeed: A Code Generation Service for IT Automation
Industry Showcase
Priyam Sahoo , Saurabh Pujar IBM Research AI, Ganesh Nalawade RED HAT, Richard Gebhardt , Louis Mandel IBM Research, USA, Luca Buratti IBM Research
Link to publication DOI Pre-print File Attached
14:15
15m
Talk
Self-planning Code Generation with Large Language Models
Journal-first Papers
Xue Jiang , Yihong Dong Peking University, Lecheng Wang Peking University, Fang Zheng Peking University, Qiwei Shang Peking University, Ge Li Peking University, Zhi Jin Peking University, Wenpin Jiao Peking University