Although Large Language Models (LLMs) have demonstrated remarkable code-generation ability, they still struggle with complex tasks. In real-world software development, humans usually tackle complex tasks through collaborative teamwork, a strategy that significantly controls development complexity and enhances software quality. Inspired by this, we present a self-collaboration framework for code generation employing LLMs, exemplified by ChatGPT. Specifically, through role instructions, 1) Multiple LLM agents act as distinct `experts’, each responsible for a specific subtask within a complex task; 2) Specify the way to collaborate and interact, so that different roles form a virtual team to facilitate each other’s work, ultimately the virtual team addresses code generation tasks collaboratively without the need for human intervention. To effectively organize and manage this virtual team, we incorporate software-development methodology into the framework. Thus, we assemble an elementary team consisting of three LLM roles (i.e., analyst, coder, and tester) responsible for software development’s analysis, coding, and testing stages. We conduct comprehensive experiments on various code-generation benchmarks. Experimental results indicate that self-collaboration code generation relatively improves 29.9%-47.1% Pass@1 compared to the base LLM agent. Moreover, we showcase that self-collaboration could potentially enable LLMs to efficiently handle complex repository-level tasks that are not readily solved by the single LLM agent.
Tue 29 OctDisplayed time zone: Pacific Time (US & Canada) change
15:30 - 16:30 | Code generation 1Journal-first Papers / Research Papers / Industry Showcase at Camellia Chair(s): Denys Poshyvanyk William & Mary | ||
15:30 15mTalk | AACEGEN: Attention Guided Adversarial Code Example Generation for Deep Code Models Research Papers Zhong Li , Chong Zhang Nanjing University, Minxue Pan Nanjing University, Tian Zhang Nanjing University, Xuandong Li Nanjing University | ||
15:45 15mTalk | Self-collaboration Code Generation via ChatGPT Journal-first Papers | ||
16:00 15mTalk | Vehicle Domain-Specific Language: Unifying Modeling and Code Generation for Low-Code Automotive Development Industry Showcase Lei Liao GAC R&D Center, Junjie Wang Institute of Software at Chinese Academy of Sciences, Zhensheng Xu GAC R&D Center, Fangwen Mu Institute of Software, Chinese Academy of Sciences, Yukun Yang GAC R&D Center | ||
16:15 15mTalk | ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code Research Papers jiafeng University of Electronic Science and Technology of China, Jiachen Liu Harbin Institute of Technology, Shenzhen, Cuiyun Gao Harbin Institute of Technology, Chun Yong Chong Huawei, Chaozheng Wang The Chinese University of Hong Kong, Shan Gao Huawei, Xin Xia Huawei Link to publication DOI Pre-print Media Attached |