DDPT: Diffusion Driven Prompt Tuning for Large Language Model Code Generation
Large Language Models (LLMs) have demonstrated remarkable capabilities in code generation. However, the quality of the generated code is heavily dependent on the structure and composition of the prompts used. Crafting high-quality prompts is a challenging task that requires significant knowledge and skills of prompt engineering. To advance the automation support for the prompt engineering for LLM-based code generation, we propose a novel solution Diffusion-Driven Prompt Tuning (DDPT) that learns how to generate optimal prompt embedding from Gaussian Noise to automate the prompt engineering for code generation. We evaluate the feasibility of diffusion-based optimization and abstract the optimal prompt embedding as a directional vector toward the optimal embedding. We use the code generation loss given by the LLMs to help the diffusion model to capture the distribution of optimal prompt embedding during training. The trained diffusion model can build a path from the noise distribution to the optimal distribution at the sampling phrase. The evaluation result enable us to assert that that DDPT helps improve the prompt optimization for code generation and diffusion-driven language modeling techniques.
Mon 28 AprDisplayed time zone: Eastern Time (US & Canada) change
16:00 - 17:30 | Generative Model EngineeringResearch and Experience Papers / Industry Talks at 208 Chair(s): Manel Abdellatif École de Technologie Supérieure | ||
16:00 15mTalk | DDPT: Diffusion Driven Prompt Tuning for Large Language Model Code Generation Research and Experience Papers Jinyang Li The University of Adelaide, Sangwon Hyun CREST, University of Adelaide, Muhammad Ali Babar School of Computer Science, The University of Adelaide | ||
16:15 15mTalk | Engineering LLM Powered Multi-agent Framework for Autonomous CloudOpsDistinguished paper Award Candidate Research and Experience Papers Kannan Parthasarathy MontyCloud, Karthik Vaidhyanathan IIIT Hyderabad, Rudra Dhar SERC, IIIT Hyderabad, India, Venkat Krishnamachari MontyCloud, Adyansh Kakran International Institute of Information Technology, Hyderabad, Sreemaee Akshathala IIIT Hyderabad, Shrikara Arun IIIT Hyderabad, Amey Karan IIIT Hyderabad, Basil Muhammed MontyCloud, Sumant Dubey MontyCloud, Mohan Veerubhotla MontyCloud | ||
16:30 15mTalk | Generating and Verifying Synthetic Datasets with Requirements Engineering Research and Experience Papers Lynn Vonderhaar Embry-Riddle Aeronautical University, Timothy Elvira Embry-Riddle Aeronautical University, Omar Ochoa Embry-Riddle Aeronautical University Pre-print | ||
16:45 15mTalk | LLM-Based Safety Case Generation for Baidu Apollo: Are We There Yet? Research and Experience Papers | ||
17:00 12mTalk | SqPal - text to SQL GenAI tool for PayPal Industry Talks | ||
17:12 18mOther | Discussion Research and Experience Papers |