The Design Space of LLM-Based AI Coding Assistants: An Analysis of 90 Systems in Academia and Industry
Over the past few years, millions of people have been using LLM-based AI tools to aid in programming, data analysis, and software engineering tasks. These AI coding assistants range from specialized tools like GitHub Copilot to general-purpose chatbots like Claude. In parallel, academics have published dozens of papers on forward-looking prototypes to expand our collective thinking beyond present-day industry trends. However, despite rapid advances in both sectors in recent years, we still lack an understanding of how their designs relate to one another and what tradeoffs are commonly made. At this key moment in 2025 when design patterns are starting to emerge, it is important to zoom out to see the forest instead of the trees. To do so, we performed the first comprehensive design analysis of 90 LLM-based AI coding assistants. We categorized the feature sets of 58 industry products and 32 academic projects, then formulated a design space that captures key variations in their user experiences. Our design space covers 10 dimensions related to UI modalities, system inputs, capabilities, and outputs. We use this design space to reveal trends in both industry and academic projects across three eras ranging from autocomplete to chat to agent-based interfaces. Lastly, to address the question of who the target users of these tools are, we present six user personas whose preferences lie in different regions of our design space: professional software engineers, HCI researchers and hobbyist programmers, UX designers, conversational programmers (e.g., product managers and marketers), data scientists, and students.
Wed 8 OctDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | AI Coding Assistants & Development ToolsResearch Papers at Duke Energy Hall Chair(s): Caitlin Kelleher Washington University in St. Louis | ||
11:00 22mTalk | Designing Human-AI Collaboration to Support Learning in Counterspeech Writing Research Papers Xiaohan Ding Virginia Tech, Kaike Ping Department of Computer Science, Virginia Tech, Uma Sushmitha Gunturi Department of Computer Science, Virginia Tech, Buse Carik Department of Computer Science, Virginia Tech, Lance T Wilhelm Department of Computer Science, Virginia Tech, Taufiq Daryanto Virginia Tech, Sophia Stil Virginia Tech, James Hawdon College of Liberal Arts and Human Sciences, Virginia Tech, Sang Won Lee Virginia Polytechnic Institute and State University, Eugenia Rho Virginia Tech Pre-print | ||
11:22 11mTalk | Programmers Without Borders: Bridging Cultures in Computer Science Study Abroad Program Research Papers | ||
11:33 11mTalk | Cracking CodeWhisperer: Analyzing Developers Interactions and Patterns During Programming Tasks Research Papers Jeena Javahar The University of British Columbia, Tanya Budhrani The Hong Kong Polytechnic University, Manaal Bascha University of British Columbia, Cleidson de Souza Federal University of Pará, Brazil, Ivan Beschastnikh The University of British Columbia, Gema Rodríguez-Pérez Department of Computer Science, Mathematics, Physics and Statistics, University of British Columbia, Okanagan Campus | ||
11:44 22mTalk | The Design Space of LLM-Based AI Coding Assistants: An Analysis of 90 Systems in Academia and Industry Research Papers Pre-print | ||
12:06 22mTalk | Designing Conversational AI to Support Think-Aloud Practice in Technical Interview Preparation for CS Students Research Papers Taufiq Daryanto Virginia Tech, Sophia Stil Virginia Tech, Xiaohan Ding Virginia Tech, Daniel Manesh Virginia Tech, Sang Won Lee Virginia Polytechnic Institute and State University, Tim Lee CodePath, Stephanie Lunn Florida International University, Sarah Rodriguez Virginia Tech, Chris Brown Virginia Tech, Eugenia Rho Virginia Tech Pre-print | ||