Exploiting Vision-Language Models in GUI Reuse
Graphical user interface (GUI) prototyping helps to clarify requirements and keep stakeholders engaged in software development. While contemporary approaches retrieve GUIs relevant to a user’s query, little support exists for the actual reuse, i.e., for using an existing GUI to create a new one. To shorten the gap, we investigate GUI-centered reuse via one of the latest artificial intelligence (AI) techniques – vision-language models (VLMs). We report an empirical study involving 73 university students working on ten GUI reuse tasks. Each task is associated with different reuse directions recommended by VLMs and by a natural language (NL) method. In addition, a focused GUI element is provided to offer a starting point for making the actual changes. Our results show that VLMs significantly outperform the NL method in making reuse recommendations, but surprisingly, the focused GUI elements are not consistently modified during reuse. With the assessments made by four experienced designers, we further offer insights into the creativity of human-reuse and AI-reuse results.
Sun 27 AprDisplayed time zone: Eastern Time (US & Canada) change
11:00 - 12:30 | Session 2: Software Product Line, Patterns and FrameworksICSR at 204 Chair(s): Gilles Perrouin Fonds de la Recherche Scientifique - FNRS & University of Namur | ||
11:00 30mPaper | A Framework for Reusable Infrastructure as Code Templates in Cloud-Native Environments ICSR File Attached | ||
11:30 30mPaper | Large Language Model assisted Transformation of Software Variants into a Software Product Line ICSR Johannes Stümpfle University of Stuttgart, Devansh Atray University of Stuttgart, Nasser Jazdi University of Stuttgart, Michael Weyrich Universiy of Stuttgart File Attached | ||
12:00 30mPaper | Exploiting Vision-Language Models in GUI Reuse ICSR Victoria Niu Walnut Hills High School, USA, Walaa Alshammari University of Cincinnati, USA, Naga Mamata Iluru University of Cincinnati, USA, Padmaja Vaishnavi Teeleti University of Cincinnati, USA, Nan Niu University of Cincinnati, Tanmay Bhowmik Mississippi State University, Jianzhang Zhang Hangzhou Normal University, China Pre-print |