FSE 2026
Sun 5 - Thu 9 July 2026 Montreal, Canada
Thu 9 Jul 2026 11:50 - 12:10 at MB 3.210 - Code Generation 2 Chair(s): Chris Brown

Large language models (LLMs) have transformed code generation, but their ability to generate code for applications with graphical user interfaces (GUIs), particularly games, remains underexplored. To explore their performance on applications with GUI, we construct a repository-aware evaluation dataset from 35 GUI applications. Prior code-generation benchmarks assess correctness using test cases, but this is insufficient for GUI applications. These applications are interactive and event-driven, and their correctness depends on stateful behavior over sequences of user actions. Consequently, evaluation should account for interaction flows and UI state transitions rather than relying solely on pass or fail test outcomes. To enable more reliable assessment beyond simple execution and unit tests, we propose Play@k, which measures whether at least one of k generated candidates yields an application that can be played end-to-end without logical errors. We further develop an LLM-based agent, PlayTester, that automates interactive evaluation by driving the GUI through task-oriented playthroughs and checking for logic violations. Through systematic evaluation, we demonstrate that 10 state-of-the-art code LLMs struggle to generate logically correct GUI applications, achieving near-zero Play@3 scores despite high compilation rates. To address these challenges, we introduce PlayCoder, a multi-agent, repository-aware framework that writes, evaluates and refines GUI application code via closed-loop control. PlayCoder substantially improves functional correctness and semantic alignment for both open-source and closed-source models, achieving up to 37.5% Exec@3 and 17.2% Play@3. Case studies show that it detects silent logic flaws missed by traditional metrics and repairs them through targeted edits. These results indicate that coupling an end-to-end GUI testing agent with repository-aware automated program repair is an effective path toward reliable GUI code generation.

Presentation Video (PlayCoder_FSE26.mp4)15.52MiB

Thu 9 Jul

Displayed time zone: Eastern Time (US & Canada) change

10:30 - 12:30
Code Generation 2Research Papers / Journal-First Paper at MB 3.210
Chair(s): Chris Brown Virginia Tech
10:30
20m
Talk
The Impact of Prompt Programming on Function-Level Code Generation
Journal-First Paper
Ranim Khojah Chalmers University of Technology and University of Gothenburg, Francisco Gomes de Oliveira Neto Chalmers | University of Gothenburg, Mazen Mohamad Chalmers | RISE - Research Institutes of Sweden, Philipp Leitner Chalmers | University of Gothenburg
Link to publication DOI
10:50
20m
Talk
GraphQLify: Automated and Type Safety-Preserving GraphQL API Adoption
Research Papers
Saleh Amareen Wayne State University, Arif Rahman Wayne State University, Sazzadur Rahaman University of Arizona, Tucson, Arizona, USA, Amiangshu Bosu Wayne State University
Pre-print
11:10
20m
Talk
Influence-Aware Bayesian-Inspired Token Reweighting for Improved Code Generation
Research Papers
YUQI ZHU Academy of Military Sciences, Ge Li Peking University, Hong Mei Peking University, Zhi Jin Peking University, Wuhan University, Jia Li Wuhan University, Qibin Zheng Advanced Institute of Big Data, Beijing, Jieyuan Zhang Academy of Military Sciences
11:30
20m
Talk
HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agent
Journal-First Paper
Jie JW Wu Michigan Technological University, USA, Fatemeh Hendijani Fard University of British Columbia, Okanagan
11:50
20m
Awards
PlayCoder: Making LLM-Generated GUI Code Playable
Research Papers
Zhiyuan Peng Shanghai Jiao Tong University, Wei Tao LightSpeed, Xin Yin Zhejiang University, Chenhao Ying Shanghai Jiao Tong University, Yuan Luo Shanghai Jiao Tong University, Yiwen Guo Unaffiliated
Pre-print File Attached