Human-AI experience in integrated development environments: a systematic literature review
The integration of Artificial Intelligence (AI) into Integrated Development Environments (IDEs) is reshaping software development, fundamentally altering how developers interact with their tools. This shift marks the emergence of Human-AI Experience in Integrated Development Environment (in-IDE HAX), a field that explores the evolving dynamics of Human-Computer Interaction in AI-assisted coding environments. Despite rapid adoption, research on in-IDE HAX remains fragmented, which highlights the need for a unified overview of current practices, challenges, and opportunities. To provide a structured overview of existing research, we conduct a systematic literature review of 90 studies, summarizing current findings and outlining areas for further investigation. We organize key insights from reviewed studies into three aspects: Impact, Design, and Quality of AI-based systems inside IDEs. Impact findings show that AI-assisted coding enhances developer productivity but also introduces challenges, such as verification overhead and over-reliance. Design studies show that effective interfaces surface context, provide explanations and transparency of suggestion, and support user control. Quality studies document risks in correctness, maintainability, and security. For future research, priorities include productivity studies, design of assistance, and audit of AI-generated code. The agenda calls for larger and longer evaluations, stronger audit and verification assets, broader coverage across the software life cycle, and adaptive assistance under user control.
Tue 7 JulDisplayed time zone: Eastern Time (US & Canada) change
14:00 - 15:30 | SE and AI 1Ideas, Visions and Reflections / Journal-First Paper / Research Papers at MB 3.210 Chair(s): Zhijie Wang Concordia University | ||
14:00 10mTalk | Bio-Aware Software Engineering for Reproducible Research Ideas, Visions and Reflections Yiming Zeng University of Connecticut, Dingbang Wang University of Connecticut, Ryan Green University of Cincinnati, Sandeep Kuttal North Carolina State University, Jinze Liu Virginia Commonwealth University, Tingting Yu University of Connecticut | ||
14:10 20mTalk | Predicting long time contributors with knowledge units of programming languages: an empirical study Journal-First Paper Md Ahasanuzzaman Queen's University, Gustavo A. Oliva Huawei Canada and Queen's University, Ahmed E. Hassan Queen’s University | ||
14:30 20mTalk | Human-AI experience in integrated development environments: a systematic literature review Journal-First Paper Agnia Sergeyuk JetBrains Research, Ilya Zakharov JetBrains Research, Ekaterina Koshchenko JetBrains Research, Mali Izadi Google & TU Delft | ||
14:50 20mTalk | Charting Uncertain Waters: A Socio-Technical Roadmap for Sustaining Open Source Communities in the Age of GenAI Journal-First Paper Zixuan Feng Virginia Commonwealth University, Reed Milewicz Sandia National Laboratories, Emerson Murphy-Hill Microsoft, Tyler Menezes CodeDay, Alexander Serebrenik Eindhoven University of Technology, Igor Steinmacher RESHAPE LAB, Northern Arizona University, USA, Anita Sarma Oregon State University Pre-print | ||
15:10 20mTalk | A Research Roadmap for Augmenting Software Engineering Processes and Software Products with Generative AI Journal-First Paper Domenico Amalfitano University of Naples Federico II, Andreas Metzger paluno – The Ruhr Institute for Software Technology, University of Duisburg-Essen, Marco Autili University of L'Aquila, Italy, Tommaso Fulcini Politecnico di Torino, Tobias Hey Karlsruhe Institute of Technology (KIT), Jan Keim Karlsruhe Institute of Technology (KIT), Patrizio Pelliccione Gran Sasso Science Institute, L'Aquila, Italy, Vincenzo Scotti Karlsruhe Institute of Technology, Anne Koziolek Karlsruhe Institute of Technology, Raffaela Mirandola Karlsruhe Institute of Technology (KIT), Andreas Vogelsang paluno – The Ruhr Institute for Software Technology, University of Duisburg-Essen | ||