Large language models (LLMs) have transformed code generation, but their ability to generate code for applications with graphical user interfaces (GUIs), particularly games, remains underexplored. To explore their performance on applications with GUI, we construct a repository-aware evaluation dataset from 35 GUI applications. Prior code-generation benchmarks assess correctness using test cases, but this is insufficient for GUI applications. These applications are interactive and event-driven, and their correctness depends on stateful behavior over sequences of user actions. Consequently, evaluation should account for interaction flows and UI state transitions rather than relying solely on pass or fail test outcomes. To enable more reliable assessment beyond simple execution and unit tests, we propose Play@k, which measures whether at least one of k generated candidates yields an application that can be played end-to-end without logical errors. We further develop an LLM-based agent, PlayTester, that automates interactive evaluation by driving the GUI through task-oriented playthroughs and checking for logic violations. Through systematic evaluation, we demonstrate that 10 state-of-the-art code LLMs struggle to generate logically correct GUI applications, achieving near-zero Play@3 scores despite high compilation rates. To address these challenges, we introduce PlayCoder, a multi-agent, repository-aware framework that writes, evaluates and refines GUI application code via closed-loop control. PlayCoder substantially improves functional correctness and semantic alignment for both open-source and closed-source models, achieving up to 37.5% Exec@3 and 17.2% Play@3. Case studies show that it detects silent logic flaws missed by traditional metrics and repairs them through targeted edits. These results indicate that coupling an end-to-end GUI testing agent with repository-aware automated program repair is an effective path toward reliable GUI code generation.
| Presentation Video (PlayCoder_FSE26.mp4) | 15.52MiB |
Thu 9 JulDisplayed time zone: Eastern Time (US & Canada) change
10:30 - 12:30 | Code Generation 2Research Papers / Journal-First Paper at MB 3.210 Chair(s): Chris Brown Virginia Tech | ||
10:30 20mTalk | The Impact of Prompt Programming on Function-Level Code Generation Journal-First Paper Ranim Khojah Chalmers University of Technology and University of Gothenburg, Francisco Gomes de Oliveira Neto Chalmers | University of Gothenburg, Mazen Mohamad Chalmers | RISE - Research Institutes of Sweden, Philipp Leitner Chalmers | University of Gothenburg Link to publication DOI | ||
10:50 20mTalk | GraphQLify: Automated and Type Safety-Preserving GraphQL API Adoption Research Papers Saleh Amareen Wayne State University, Arif Rahman Wayne State University, Sazzadur Rahaman University of Arizona, Tucson, Arizona, USA, Amiangshu Bosu Wayne State University Pre-print | ||
11:10 20mTalk | Influence-Aware Bayesian-Inspired Token Reweighting for Improved Code Generation Research Papers YUQI ZHU Academy of Military Sciences, Ge Li Peking University, Hong Mei Peking University, Zhi Jin Peking University, Wuhan University, Jia Li Wuhan University, Qibin Zheng Advanced Institute of Big Data, Beijing, Jieyuan Zhang Academy of Military Sciences | ||
11:30 20mTalk | HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent Journal-First Paper Jie JW Wu Michigan Technological University, USA, Fatemeh Hendijani Fard University of British Columbia, Okanagan | ||
11:50 20mAwards | PlayCoder: Making LLM-Generated GUI Code Playable Research Papers Zhiyuan Peng Shanghai Jiao Tong University, Wei Tao LightSpeed, Xin Yin Zhejiang University, Chenhao Ying Shanghai Jiao Tong University, Yuan Luo Shanghai Jiao Tong University, Yiwen Guo Unaffiliated Pre-print File Attached | ||