Prompts as Software Engineering Artifacts: A Research Agenda and Preliminary Findings
Developers now routinely interact with large language models (LLMs) to support a range of software engineering (SE) tasks. This prominent role positions prompts as potential SE artifacts that, like other artifacts, may require systematic development, documentation, and maintenance. However, little is known about how prompts are actually used and managed in LLM-integrated workflows, what challenges practitioners face, and whether the benefits of systematic prompt management outweigh the associated effort. To address this gap, we propose a research programme that (a) characterizes current prompt practices, challenges, and influencing factors in SE; (b) analyzes prompts as software artifacts, examining their evolution, traceability, reuse, and the trade-offs of systematic management; and (c) develops and empirically evaluates evidence-based guidelines for managing prompts in LLM-integrated workflows. As a first step, we conducted an exploratory survey with 74 software professionals from six countries to investigate current prompt practices and challenges. The findings reveal that prompt usage in SE is largely ad-hoc: prompts are often refined through trial-and-error, rarely reused, and shaped more by individual heuristics than standardized practices. These insights not only highlight the need for more systematic approaches to prompt management but also provide the empirical foundation for the subsequent stages of our research programme.
| Prompt as Software Engineering Artifacts: A Research Agenda and Preliminary Findings (PROFES-2025-Prompts.pdf) | 1.11MiB |
Tue 2 DecDisplayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change
11:30 - 13:00 | Artificial Intelligence and Analytics in Software EngineeringIndustry Papers / Research Papers / Short Papers and Posters at Sala degli Affreschi (Fresco Room) Chair(s): Antonio Martini University of Oslo, Norway | ||
11:30 15mTalk | Generating Business Process Models with Open Source Large Language Models using Instruction Tuning Research Papers Gökberk Çelikmasat Boğaziçi University, Atay Özgövde Boğaziçi University, Fatma Başak Aydemir Utrecht University | ||
11:45 15mTalk | Application of Large Language Models in Product Management: A Systematic Literature Review Research Papers Vitor Mori Eindhoven University of Technology (TU/e), Jan Bosch Chalmers University of Technology, Helena Holmström Olsson Malmö University | ||
12:00 15mTalk | Towards Understanding Team Congestion in Large-Scale Software Development Research Papers Javier Gonzalez-Huerta Blekinge Institute of Technology, Ehsan Zabardast Nordea / Blekinge Institute of Technology File Attached | ||
12:15 10mTalk | A Small Dataset May Go a Long Way: Process Duration Prediction in Clinical Settings Industry Papers File Attached | ||
12:25 7mTalk | Prompts as Software Engineering Artifacts: A Research Agenda and Preliminary Findings Short Papers and Posters Hugo Villamizar fortiss GmbH, Jannik Fischbach Netlight Consulting GmbH and fortiss GmbH, Alexander Korn University of Duisburg-Essen, Andreas Vogelsang paluno – The Ruhr Institute for Software Technology, University of Duisburg-Essen, Daniel Mendez Blekinge Institute of Technology and fortiss DOI File Attached | ||
12:32 7mTalk | MAPS-AI – A Tool for AI-Assisted Model-Driven Generation of IT Project Plan and Scope Short Papers and Posters Oksana Nikiforova Riga Technical University, Rihards Bobkovs Riga Technical University, Megija Krista Miļūne Riga Technical University, Kristaps Babris Riga Technical University, Oscar Pastor Universitat Politecnica de Valencia, Jānis Grabis Riga Technical University | ||
12:39 7mTalk | Cost of Artificial Intelligence in Finnish Software Companies: A Survey Short Papers and Posters Antti Klemetti University of Helsinki, Anssi Sorvisto University of Jyväskylä, Mikko Raatikainen University of Helsinki, Jukka K. Nurminen University of Helsinki File Attached | ||
12:46 7mTalk | Exploring the Performance of ML Model Size for Classification in Relation to Energy Consumption Short Papers and Posters Andreas Bexell Ericsson, Lo Heander Lund University, Emma Söderberg Lund University, Sigrid Eldh Ericsson AB, Mälardalen University, Carleton University, Per Runeson Lund University | ||
12:53 7mTalk | On the Use of Agentic Coding Manifests: An Empirical Study of Claude Code Short Papers and Posters Worawalan Chatlatanagulchai Kasetsart University, Kundjanasith Thonglek Kasetsart University, Brittany Reid Nara Institute of Science and Technology, Yutaro Kashiwa Nara Institute of Science and Technology, Pattara Leelaprute Kasetsart University, Arnon Rungsawang Kasetsart University, Bundit Manaskasemsak Kasetsart University, Hajimu Iida Nara Institute of Science and Technology File Attached | ||