End-to-End Model Generation with Large Language Models for Adaptive IoT Application Deployment
The deployment of AI-powered applications on resource-constrained edge devices presents a significant software engineering challenge. While Pruning and Neural Architecture Search (NAS) have shown promise in optimizing model efficiency, their application in edge device deployment is often limited by low levels of automation. In this paper, we introduce LLM-based Adaptive Model Generation (LAMDA), a novel framework that tackle the challenge of adaptive AI deployment as an automated model generation problem. LAMDA empowers an LLM to perform end-to-end design, refinement, and optimization of DNNs to meet the specific hardware constraints. Our approach makes two primary contributions. First, we introduce a serialization technique that transforms complex DNN computation graphs into a structured textual representation that makes model architectures comprehensible and manipulable by an LLM. Further, to ground the generation process in real-world hardware constraints, we integrate a feedback-driven optimization loop. This loop leverages an empirical performance model, trained to correlate architectural patterns with on-device latency, enabling the LLM to reason about and optimize for non-functional requirements. To mitigate architectural ``hallucinations'', we incorporate context management and validation to ensure valid generation. We evaluate LAMDA through extensive experiments on public benchmarks and real-world edge devices. The results demonstrate that our framework can autonomously generate and adapt DNNs that satisfy deployment-specific accuracy and latency constraints, significantly advancing the state-of-the-art in automated software adaptation for the AI-enabled edge devices.
Thu 16 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | AI for Software Engineering 10Research Track / Journal-first Papers at Asia I Chair(s): Fabio Marcos De Abreu Santos Colorado State University, USA | ||
11:00 15mTalk | FlipFlop: A Static Analysis-based Energy Optimization Framework for GPU Kernels Research Track Saurabhsingh Rajput Dalhousie University, Alexander Brandt Dalhousie University, Vadim Elisseev IBM, Tushar Sharma Dalhousie University Pre-print | ||
11:15 15mTalk | Portable Power Modeling with Transfer Learning on JVM-Based Applications Research Track | ||
11:30 15mTalk | End-to-End Model Generation with Large Language Models for Adaptive IoT Application Deployment Research Track ZHENYU WEN Zhejiang University of Technology, Jintao Feng Zhejiang University of Technology, Yao Nanjie Zhejiang University of Technology, Di Wu University of Central Florida, Cong Wang Zhejiang University, China, Mincheng Wu Zhejiang University of Technology, Jianbin Qin Shenzhen Institute of Computing Sciences, Shenzhen University, Shibo He Zhejiang University | ||
11:45 15mTalk | Efficient and Green Large Language Models for Software Engineering: Literature Review, Vision, and the Road Ahead Journal-first Papers | ||
12:00 15mTalk | An Empirical Study of Knowledge Distillation for Code Understanding Tasks Research Track Ruiqi Wang Harbin Institute of Technology, Shenzhen, Zezhou Yang , Cuiyun Gao Harbin Institute of Technology, Shenzhen, Xin Xia Zhejiang University, Qing Liao Harbin Institute of Technology Pre-print | ||
12:15 15mTalk | Generating Energy-Efficient Code via Large-Language Models - Where are we now? Research Track Radu Apsan Vrije Universiteit Amsterdam, The Netherlands, Vincenzo Stoico Vrije Universiteit Amsterdam, Michel Albonico Federal University of Technology, ParanĂ¡ (UTFPR), Rudra Dhar IIIT Hyderabad, Karthik Vaidhyanathan IIIT Hyderabad, Ivano Malavolta Vrije Universiteit Amsterdam Pre-print Media Attached | ||