Prompting Without Principles: Are Students Transferring Software Engineering Knowledge to LLM Use?
This program is tentative and subject to change.
Generative AI (GenAI), in particular large language models (LLMs), have rapidly become part of software engineers’ toolboxes. While the software engineering community has actively debated efficacy and extent of use of GenAI tools, little is known about how early- career developers apply these tools and transfer their classroom training into practice. To address this gap, we conducted a con- trolled study with graduate students in a top-tier Master of Software Engineering program who were enrolled in a software refactoring course. The study compared how students solved a library upgrade problem using an ad hoc non-LLM approach versus an automated approach supported by LLMs. Our findings show that students of- ten failed to apply disciplined software engineering practices when relying on LLMs, simply feeding them error text from builds rather than using them to understand issues and explore solutions. Our results highlight a dual challenge for educators: reinforcing foun- dational software engineering skills and their transfer to GenAI use, while also introducing new competencies such as effective prompting and navigating LLM-driven workflows.
This program is tentative and subject to change.
Thu 16 AprDisplayed time zone: Brasilia, Distrito Federal, Brazil change
11:00 - 12:30 | |||
11:00 15mTalk | Exploring the Community of Inquiry in Online Computing Education: Student Perceptions and Opportunities for Generative AI Software Engineering Education and Training (SEET) | ||
11:15 15mTalk | Prompting Without Principles: Are Students Transferring Software Engineering Knowledge to LLM Use? Software Engineering Education and Training (SEET) Leonardo Da Silva Sousa Carnegie Mellon University, USA, Ipek Ozkaya Carnegie Mellon University, James Ivers Carnegie Mellon University, Celina Cywinska Carnegie Mellon University, Bingyu Xie Carnegie Mellon University, Mena Kostial Carnegie Mellon University Software Engineering Institute, Tapajit Dey Carnegie Mellon University Software Engineering Institute, Robert Edman Carnegie Mellon Software Engineering Institute | ||
11:30 15mTalk | "Can you feel the vibes?": An exploration of novice programmer engagement with vibe coding Software Engineering Education and Training (SEET) Kiev Gama Universidade Federal de Pernambuco, Filipe Calegario Universidade Federal de Pernambuco, Victoria Jackson University of Southampton, Alexander Nolte Eindhoven University of Technology, Luiz Morais Universidade Federal de Pernambuco, Vinicius Cardoso Garcia Universidade Federal de Pernambuco | ||
11:45 15mTalk | The Clash of Codes: From Peer-to-Peer Duplication to AI-Generation in Introductory Programming Assignments Software Engineering Education and Training (SEET) Jose Maria Zuzarte Reis Claver Vrije Universiteit Amsterdam, i Mahbod Tajdin Vrije Universiteit Amsterdam, Mauricio Verano Merino Vrije Universiteit Amsterdam | ||
12:00 15mTalk | AI-Assisted Code Review as a Scaffold for Code Quality and Self-Regulated Learning: An Experience Report Software Engineering Education and Training (SEET) Eduardo Araujo Oliveira The University of Melbourne, Michael Fu The University of Melbourne, Patanamon Thongtanunam University of Melbourne, Sonsoles López-Pernas University of Eastern Finland, Mohammed Saqr University of Eastern Finland | ||
12:15 15mTalk | Amplifiers or Equalizers? A Longitudinal Study of LLM Evolution in Software Engineering Project-Based Learning Software Engineering Education and Training (SEET) | ||