Multi-language Software Development in the LLM Era: Insights from Practitioners’ Conversations with ChatGPT
Non-trivial software systems are commonly developed using more than a single programming language, averaging 5 different languages according to recent studies. However, multi-language development is not straightforward. Developers need to be proficient in each language and also know the best integration practices between languages. Nowadays, tools powered by Large Language Models (LLMs), such as ChatGPT, have been shown to successfully assist practitioners in several aspects of software development. This paper reports a preliminary study aimed to investigate to what extent ChatGPT is being used in multi-language development scenarios. Hence, we leveraged DevGPT, a dataset of conversations between software practitioners and ChatGPT which also includes links to GitHub repositories from which the conversations originated. In total, we studied data from 3,584 conversations, comprising a total of 18,862 code snippets suggested by ChatGPT. Our analyses show that only 18.33% of the code snippets suggested by ChatGPT are written in the same programming language as the primary language in the repository where the conversation was shared. An in-depth analysis showcased interesting remarks. While we observed expected scenarios, such as 31.54% of JavaScript snippets being suggested in CSS repositories, we also unveiled surprising ones, such as Python snippets being largely suggested in C++ repositories. After a qualitative open card sorting of the conversations involving multiple languages, we found that in 70% of them developers were asking for coding support while in 57% developers used ChatGPT as a tool to generate code automatically. Our initial results indicate that not only are LLM-based tools being used in multi-language development but also showcase the most common contexts in which such tools are assisting developers.
Fri 25 OctDisplayed time zone: Brussels, Copenhagen, Madrid, Paris change
14:00 - 15:30 | Large language models in software engineering IIESEM Emerging Results, Vision and Reflection Papers Track / ESEM IGC at Telensenyament (B3 Building - 1st Floor) Chair(s): Claudio Di Sipio University of l'Aquila | ||
14:00 15mVision and Emerging Results | Debugging with Open-Source Large Language Models: An Evaluation ESEM Emerging Results, Vision and Reflection Papers Track Yacine Majdoub IResCoMath Lab, University of Gabes, Eya Ben Charrada IResCoMath Lab, University of Gabes Link to publication DOI Pre-print | ||
14:15 15mVision and Emerging Results | Multi-language Software Development in the LLM Era: Insights from Practitioners’ Conversations with ChatGPT ESEM Emerging Results, Vision and Reflection Papers Track Lucas Almeida Aguiar State University of Ceará, Matheus Paixao State University of Ceará, Rafael Carmo Federal University of Ceará, Edson Soares Instituto Atlantico & State University of Ceara (UECE), Antonio Leal State University of Ceará, Matheus Freitas State University of Ceará, Eliakim Gama State University of Ceará | ||
14:30 15mVision and Emerging Results | Exploring LLM-Driven Explanations for Quantum Algorithms ESEM Emerging Results, Vision and Reflection Papers Track Giordano d'Aloisio University of L'Aquila, Sophie Fortz King's College London, Carol Hanna University College London, Daniel Fortunato INESC-ID, University of Porto, Avner Bensoussan King's College London, Eñaut Mendiluze Usandizaga Simula Research Laboratory, Norway, Federica Sarro University College London Pre-print | ||
14:45 15mIndustry talk | Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysis ESEM IGC Matteo Esposito University of Oulu, Francesco Palagiano Multitel di Lerede Alessandro & C. s.a.s., Valentina Lenarduzzi University of Oulu, Davide Taibi University of Oulu Pre-print | ||
15:00 15mVision and Emerging Results | Detecting Code Smells using ChatGPT: Initial Insights ESEM Emerging Results, Vision and Reflection Papers Track Luciana L. Silva Federal University of Minas Gerais, Janio R. Silva IFMG, João Eduardo Montandon Universidade Federal de Minas Gerais (UFMG), Marcus Andrade IFMG, Marco Tulio Valente Federal University of Minas Gerais, Brazil | ||
15:15 15mIndustry talk | ChatGPT’s Potential in Cryptography Misuse Detection: A Comparative Analysis with Static Analysis Tools ESEM IGC |